Nanoparticles (NPs) hold great promises for targeted disease diagnosis and therapy. Despite considerable progress in biomimetic design of NP-bioconjugates, the roles of NP size and shape in endocytosis are still not fully understood. Using an efficient coarse-grained molecular dynamics (CGMD) model, we simulate receptor-mediated endocytosis of NPs of various sizes and shapes. Our simulations demonstrate that both NP size and shape modulate the kinetics of endocytosis. For spherical NPs, there exists an optimal size at which endocytosis takes the shortest time. For a spherocylindrical NP with the initial upright docking position on the membrane plane, endocytosis proceeds through a laying-down-then-standing-up sequence. A free energy analysis reveals that NP size primarily determines whether endocytosis can complete, while NP shape breaks the symmetry of curvature energy landscape and hence dictates the endocytic pathway and the angle of entry. The findings shed light on the rational design of NP-based diagnostic and therapeutic agents with improved cellular targeting.