In this paper, a new branch-and-bound algorithm based on the Lagrangian dual relaxation and continuous relaxation is proposed for discrete multi-factor portfolio selection model with roundlot restriction in financial optimization. This discrete portfolio model is of integer quadratic programming problems. The separable structure of the model is investigated by using Lagrangian relaxation and dual search. Computational results show that the algorithm is capable of solving real-world portfolio problems with data from US stock market and randomly generated test problems with up to 120 securities.