Coral recruitment is important in sustaining coral reef ecosystems and contributing to their recovery after disturbances. Despite widespread acceptance that crustose coralline algae (CCA) positively influence coral recruitment success, especially by enhancing coral settlement and early post-settlement stages, there are no experimental data on the effects of CCA species on late post-settlement survival and growth of corals. This study tested the impact of four common, thick-crusted CCA species from two habitats (exposed and subcryptic) on the survival and growth of two recruit size categories of the coral genus Pocillopora. Coral recruits and CCA were transplanted adjacent to each other using epoxy in Petri dishes directly attached to the reef substratum on the forereef of Moorea (French Polynesia) in a 1-year field experiment. In the subcryptic habitat, survival of small-sized recruits adjacent to subcryptic CCA (0–5%) was lower than adjacent to dead CCA (35%), while in the exposed habitat, survival of small-sized recruits adjacent to exposed CCA (20–25%) was higher than adjacent to dead CCA (5%). None of the CCA species affected the survival of large-sized recruits within exposed or subcryptic habitats. However, the growth of large-sized recruits adjacent to subcryptic CCA was lower than adjacent to dead CCA. Recruits adjacent to exposed CCA died less from competition with turf algae relative to dead CCA, while recruits adjacent to subcryptic CCA frequently died from overgrowth by CCA. These results suggest that, in subcryptic habitats, CCA can reduce the survival and/or growth of coral recruits via direct competitive overgrowth, while in exposed habitats, they can enhance coral recruit survival by alleviating competition with turf algae. Importantly, our study demonstrates that not all CCA species are beneficial to the survival and growth of coral recruits and that there is considerable variability in both the outcome and process of competition between CCA and corals.