Study of Neoadjuvant PARP Inhibition Followed by Radical Prostatectomy in Patients With Unfavorable Intermediate-Risk or High-Risk Prostate Cancer With Select HRR Gene Alterations

Inclusion Criteria: Subject must meet all of the following applicable inclusion criteria to participate in this study: * Written informed consent and HIPAA authorization for release of personal health information prior to registration. * Age ≥ 18 years at the time of consent. * T stage 1-3 prostatic adenocarcinoma per AJCC staging manual Ed8. * Histologically confirmed adenocarcinoma of the prostate without histological variants comprising \>50% of the sample. Patients with intraductal carcinoma are eligible. * Must have 3 core biopsies involved with cancer (a minimum of 6 core biopsies must be obtained). Prostate biopsy must be within 7 months from registration. Less than 3 core biopsies are allowed if the patient has \>1 cm or T3 disease on magnetic resonance imaging (MRI). * Localized unfavorable intermediate or high-risk prostate cancer patients. Patients must have at least one of the following features: * Gleason ≥ 4+3 (grade group 3, 4, 5) OR * PSA \> 20 ng/dL OR * T3 disease NOTE: Patients with intraductal carcinoma are eligible independent of Gleason score, PSA and T stage. * Must have evidence of germline or somatic BRCA1/2, PALB2, RAD51B, RAD51C, RAD51D, RAD54L2, BARD1, FANCA, BRIP1, CHEK2, ATM, and CDK12 gene alteration via standard of care CLIA based assay detection. Testing will be confirmed centrally but results of central testing not required for enrollment. * No evidence of metastatic disease as determined by radionuclide bone scan and CT/MRI. Lymph nodes must be less than 20 mm in the short (transverse) axis. * Participants must be candidates for RP and considered surgically resectable by urologic evaluation. * ECOG Performance Status of 0-1 within 28 days prior to registration. * Demonstrate adequate organ function as defined below. All screening labs to be obtained within 28 days prior to registration. * White blood cell count ≥ 3,000/mcL * Absolute neutrophil count ≥ 1,500/mcL * Hemoglobin ≥ 10 g/dL with no transfusion support in the past 28 days * Platelets ≥ 100,000/mcL * Aspartate aminotransferase, alanine aminotransferase ≤ 2.5×ULN, and total bilirubin ≤ 1.5 x Institutional upper limit of normal * Calculated creatinine clearance ≥ 51 mL/min based on Cockcroft-Gault formula or 24 hour urine. NOTE: See the protocol for Cockcroft-Gault formula or 24 hour urine. * Life expectancy≥ 16 weeks. * Subjects must use a condom plus spermicide beginning prior to treatment Cycle 1 Day 1, during treatment and for 3 months after the last dose of olaparib when having sexual intercourse with a pregnant woman or with a woman of childbearing potential. Female partners of male patients should also use a highly effective form of contraception if they are of childbearing potential. See protocol for additional details. * As determined by the enrolling physician or protocol designee, ability of the subject to understand and comply with study procedures for the entire length of the study. Exclusion Criteria: Subjects meeting any of the criteria below may not participate in the study: * Active infection requiring systemic therapy. * Prior treatments not allowed: hormone therapy for prostate cancer including orchiectomy, antiandrogens (including first-generation antiandrogens, enzalutamide, apalutamide and others), CYP17 inhibitors (including abiraterone, TAK-700, galeterone, ketoconazole, and others), estrogens and radiation therapy. Prior bicalutamide is allowed if taken for \< 4 weeks prior to registration and there is a washout period of 2 weeks prior to the initiation of study treatment. LHRH agonist/antagonist therapy is allowed if begun within 4 weeks of registration. Prior 5-alpha reductase inhibitors are allowed but require a washout period of 2 weeks to initiation of study treatment. * Prior treatment with a PARP inhibitor. * Clinically significant acute infection requiring systemic antibacterial, antifungal, or antiviral therapy including: * tuberculosis (clinical evaluation that includes clinical history, physical examination, and radiographic findings, and TB testing in line with local practice). * Known active hepatitis infection, positive hepatitis C antibody, hepatitis B virus surface antigen or hepatitis B virus core antibody at screening. Testing is not required unless there was a prior known positive hepatitis B or C test or hepatitis is suspected at screening. Active hepatitis B virus (HBV) is defined by a known positive HBV surface antigen (HBsAg) result. For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated. Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load. * Known to have tested positive for human immunodeficiency virus (HIV) unless currently on effective anti-retroviral therapy with an undetectable viral load within 6 months. * Severe hepatic impairment (Child-Pugh Class C). * Patients unable to swallow orally administered medication and patients with gastrointestinal disorders likely to interfere with absorption of the study medication. * Pre-existing condition that warrants long-term corticosteroid use greater than the equivalent of 10 mg prednisone daily. Physiologic replacement is permitted. Topical, intra-articular steroids or inhaled corticosteroids are permitted. * Active cardiac disease, defined as: * Myocardial infarction within 6 months of study treatment. * Uncontrolled angina within 3 months of study treatment. * Congestive heart failure New York Heart Association (NYHA) class 3 or 4, or subjects with history of congestive heart failure NYHA class 3 or 4 in the past, or history of anthracycline or anthracenedione (mitoxantrone) treatment, unless an echocardiogram performed within 3 months of the screening visit results in a left ventricular ejection fraction that is ≥ 45%. * History of clinically significant ventricular arrhythmias (e.g., ventricular tachycardia, ventricular fibrillation, torsade de pointes) * Other clinically significant cardiovascular disease within 6 months of registration. * Uncontrolled, potentially reversible cardiac conditions, as judged by the investigator (eg., unstable ischemia, uncontrolled symptomatic arrhythmia, congestive heart failure, QTcF prolongation \>500 ms, electrolyte disturbances, etc.), or patients with congenital long QT syndrome. * Individuals with a history of another malignancy are not eligible if the cancer is under active treatment or the cancer can be seen on radiology scans. * Major surgery within 4 weeks from start of treatment. Subjects must have recovered from any effects as the surgery as assessed by investigator discretion. * Treatment with any investigational drug within 28 days prior to registration. * Persistent toxicities Grade \> 2 caused by previous cancer therapy (per Common Terminology Criteria for Adverse Event (CTCAE)). * Patients with myelodysplastic syndrome/acute myeloid leukemia or with features suggestive of MDS/AML. * Patients considered a poor medical risk due to a serious, uncontrolled medical disorder, non-malignant systemic disease or active, uncontrolled infection. Examples include but are not limited to, uncontrolled ventricular arrhythmia, recent (within 3 months) myocardial infarction, uncontrolled major seizure disorder, unstable spinal cord compression, superior vena cava syndrome, extensive interstitial bilateral lung disease on High Resolution Computed Tomography (HRCT) scan or any psychiatric disorders that prohibits obtaining informed consent. * Patients receiving any systemic chemotherapy or radiotherapy (except for palliative reasons) within 3 weeks prior to study treatment. * Concomitant use of known strong CYP3A inhibitors. The required washout period prior to starting olaparib is 2 weeks. * Concomitant use of known strong or moderate CYP3A inducers. The required washout period prior to starting olaparib is 5 weeks for enzalutamide or phenobarbital and 3 weeks for other agents. * Previous allogenic bone marrow transplant or double umbilical cord blood transplantation (dUCBT). * Whole blood transfusions in the last 120 days prior to entry to the study (packed red blood cells and platelet transfusions are acceptable, for timing refer to the protocol). * Patients with a known hypersensitivity to olaparib or any of the excipients of the product.