The Lyot Project Direct Imaging Survey of Substellar Companions: Statistical Analysis and Information from Nondetections

The Lyot project used an optimized Lyot coronagraph with extreme adaptive optics at the 3.63 m Advanced Electro-Optical System telescope to observe 86 stars from 2004 to 2007. In this paper, we give an overview of the survey results and a statistical analysis of the observed nondetections around 58 of our targets to place constraints on the population of substellar companions to nearby stars. The observations did not detect any companion in the substellar regime. Since null results can be as important as detections, we analyzed each observation to determine the characteristics of the companions that can be ruled out. For this purpose, we use a Monte Carlo approach to produce artificial companions and determine their detectability by comparison with the sensitivity curve for each star. All the non-detection results are combined using a Bayesian approach and we provide upper limits on the population of giant exoplanets and brown dwarfs for this sample of stars. Our nondetections confirm the rarity of brown dwarfs around solar-like stars and we constrain the frequency of massive substellar companions (M>40 M{sub J}) at orbital separation between and 10 and 50 AU to be {approx_lt}20%.