Sacituzumab Govitecan With or Without Atezolizumab Immunotherapy in Rare Genitourinary Tumors (SMART) Such as High Grade Neuroendocrine Carcinomas, Adenocarcinoma, and Squamous Cell Bladder/Urinary Tract Cancer, Renal Medullary Carcinoma and Penile C...

* INCLUSION CRITERIA: * Participants must have histologically confirmed diagnosis of a locally advanced unresectable or metastatic non-prostate genitourinary (GU) tumor of the following histologies: * HGNEC, including, but not limited to, small cell carcinoma and large cell neuroendocrine carcinoma of the bladder or urinary tract * Squamous cell carcinoma of the bladder or urinary tract * Primary adenocarcinoma of the bladder or urinary tract (urachal or non-urachal) * Renal medullary carcinoma * Squamous cell carcinoma of the penis Note: For the purposes of enrollment, the urinary tract is defined as the renal pelvis, ureter, bladder, and urethra. * Pre-study treatment tissue availability (sufficient tissue for approximately 25 unstained slides is mandatory for enrollment. If tissue is determined to be insufficient/unsuitable, a fresh biopsy prior to study therapy will be required. * Locally advanced unresectable or metastatic disease. Participants who have received prior treatment must have evidence of progressive disease (PD; i.e., defined as new or progressive lesions evident on cross-sectional imaging). * Prior treatment as follows: * Cohort A: Participants must have received prior ICIs (PD-1 or PD-L1) or be ineligible for treatment with ICIs. * For Cohort B: Participants must be ICI naive but eligible to receive them. * Participants must have measurable disease, per RECIST 1.1. * Age \>= 18 years. * Eastern Cooperative Oncology Group (ECOG) performance status \<= 1 (Karnofsky \>= 70%. * Adequate organ and marrow function as defined below: * Hemoglobin (Hgb) \>= 9.0 g/dL * Absolute neutrophil count (ANC) \>= 1,500/mcL * Platelets \>= 100,000/mcL * Total bilirubin \<= 1.5 x upper limit of normal (ULN) (\<= 3 x ULN in participants with known/suspected Gilbert s disease) * AST/ ALT \<= 2.5 x ULN (or \<= 5 x ULN if considered to be related to liver metastases by the PI) * Serum creatinine \<= 2 x ULN or creatinine clearance \>= 30 ml/min/1.73 m\^2 (glomerular filtration rate \[GFR\] may be used in place of CrCl. Creatinine clearance or eGFR should be calculated per institutional standard) * Alkaline phosphatase \<= 2.5 x ULN (or \<= 5 x ULN if considered to be related to liver or bone metastases by the PI) * Serum albumin \>= 25g/L * For participants not receiving therapeutic anticoagulation: international normalized ratio (INR) or activated partial thromboplastin time (aPTT) \<= 1.5 x ULN * For participants receiving therapeutic anticoagulation: stable anticoagulant regimen * Participants may have received any number of prior anti-cancer treatments or be treatment na(SqrRoot) ve (except for participants with HGNEC of the bladder/urinary tract cancer, whom must have received a platinum-based combination regimen either as neoadjuvant, adjuvant or first-line treatment in the locally advanced/metastatic setting). * Treated central nervous system (CNS) lesions, provided that all of the following criteria are met: * Measurable disease, per RECIST v1.1, must be present outside the CNS. * The participant has no history of intracranial hemorrhage or spinal cord hemorrhage. * The participant has not undergone stereotactic radiotherapy within 1 week prior to initiation of study treatment, whole-brain radiotherapy (WBXRT) within 2 weeks prior to initiation of study treatment, or neurosurgical resection within 4 weeks prior to initiation of study treatment. * The participant has no ongoing requirement for corticosteroids as therapy for CNS disease. * The participant may be receiving anti-convulsant therapy if appropriate and the dose is considered stable. Prior treatment as follows: * Prior radionuclide treatment must have a washout period of at least 6 weeks prior to the first dose of study treatment. * Prior treatment with chemotherapy must have a washout period of 2 weeks prior to the first dose of study treatment. * Prior treatment with non-CNS-directed radiotherapy must have a washout period of 2 weeks prior to the first dose of study treatment (except palliative bone-directed radiotherapy which does not require any washout). * Prior treatment with a small molecule kinase inhibitor must have a washout period of at least 2 weeks or five half-lives of the compound or active metabolites, prior to the first dose of study treatment. * Prior treatment with systemic immunostimulatory agents (including, but not limited to, interferon and interleukin 2 \[IL-2\]) must have a washout period of at least 4 weeks or 5 half-lives of the drug (whichever is longer) prior to the first dose of study treatment. * Major surgical procedure, other than for diagnosis, must not occur within 4 weeks prior to the first dose of study treatment. * Prior treatment with systemic immunosuppressive medication (including, but not limited to, corticosteroids, cyclophosphamide, azathioprine, methotrexate, thalidomide, and anti-TNF-alpha agents) must have a washout period of at least 2 weeks prior to initiation of study treatment or anticipation of need for systemic immunosuppressive medication during study treatment, with the following exceptions: * Participants who received acute, low-dose systemic immunosuppressant medication or a one-time pulse dose of systemic immunosuppressant medication (e.g., 48 hours of corticosteroids for a contrast allergy) are eligible for the study. * Participants who received mineralocorticoids (e.g., fludrocortisone), corticosteroids for chronic obstructive pulmonary disease (COPD) or asthma, or low-dose corticosteroids for orthostatic hypotension or adrenal insufficiency are eligible for the study. * FDA-approved hormonal therapy for the treatment or prevention of other malignancies (e.g., breast cancer, prostate cancer) are allowed to be continued where in the opinion of the treating investigator stopping such therapies may increase the risk of disease progression. Potential drug-drug interactions with the hormonal agent will be assessed by the treating investigator prior to enrollment and hormonal agents that inhibit or induce UGT1A1 will be excluded while on trial. * Human immunodeficiency virus (HIV)-infected participants are eligible if on stable dose of highly active antiretroviral therapy (HAART), a CD4 count \>= 200 cells/microL, and an undetectable viral load. * Hepatitis B virus (HBV) positive participants are eligible if they have been treated or are on an appropriate course of antivirals at study entry and with planned monitoring and management according to appropriate guidance. For previously treated patients or those with prior infection that has been cleared, prophylaxis is permitted, and hepatology consultation recommended. * Participants with a history of hepatitis C virus (HCV) infection (i.e., positive HCV antibody test) must have been treated and cured (negative HCV RNA test at screening). Participants with HCV infection who are currently on treatment are eligible if they have an undetectable HCV viral load. * Individuals of child-bearing potential (IOCBP) and individuals able to father a child must agree to use an effective method of contraception as follows: * IOCBP must agree to use one (1) highly effective methods of contraception (e.g., intrauterine device \[IUD\], hormonal, surgical sterilization) prior to study entry, for the duration of study participation, and for up to 6 months after discontinuation of the study drug(s). Participants must refrain from donating eggs during this same period. * Individuals able to father children must agree to use an effective method of contraception (barrier, surgical sterilization) for the duration of the study treatment and up to 6 months after the last dose of the study drug(s) and must refrain from donating sperm during this same period. * Nursing participants must discontinue nursing and/or not begin nursing until 1 month after the last dose of study drug(s). * Ability of participants to understand and the willingness to sign a written informed consent document. EXCLUSION CRITERIA: * History of severe hypersensitivity or allergic reactions attributed to compounds of similar chemical or biologic composition to SG, SN-38, irinotecan, or atezolizumab, or hypersensitivity to Chinese hamster ovary cell products. * Symptomatic or untreated brain/CNS metastases. * Positive serum or urine Beta-human chorionic gonadotropin (Beta-hCG) test at screening. * Participants unwilling to accept blood products as medically indicated. * For Cohort B: Active or history of autoimmune disease or immune deficiency that might recur, which might affect vital organ function or require immune suppressive treatment including systemic corticosteroids, when receiving atezolizumab. These conditions include myasthenia gravis, myositis, autoimmune hepatitis, systemic lupus erythematosus, rheumatoid arthritis, inflammatory bowel disease, antiphospholipid antibody syndrome, Wegener granulomatosis, Sjogren syndrome, Guillain-Barre syndrome, or multiple sclerosis, with the following exceptions: * Participants with a history of autoimmune-related hypothyroidism who are on thyroid-replacement hormone are eligible for the study. * Participants with controlled Type 1 diabetes mellitus who are on an insulin regimen are eligible for the study. * Participants with eczema, psoriasis, lichen simplex chronicus, or vitiligo with dermatologic manifestations only (e.g., participants with psoriatic arthritis are excluded) are eligible for the study provided all of following conditions are met: * Rash must cover \< 10% of body surface area. * Disease is well controlled at baseline and requires only low-potency topical corticosteroids. * There has been no occurrence of acute exacerbations of the underlying condition requiring psoralen plus ultraviolet A radiation, methotrexate, retinoids, biologic agents, oral calcineurin inhibitors, or high-potency or oral corticosteroids within the previous 12 months. * History of idiopathic pulmonary fibrosis, organizing pneumonia (e.g., bronchiolitis obliterans), drug-induced pneumonitis, or idiopathic pneumonitis, or evidence of active pneumonitis on screening chest computed tomography (CT) scan. History of radiation pneumonitis in the radiation field (fibrosis) is permitted. * Anticipation of need for a major surgical procedure during the study. * Prior allogeneic stem cell or solid organ transplantation. * Treatment with a live, attenuated vaccine within 4 weeks prior to initiation of study treatment, or anticipation of need for such a vaccine during SG or atezolizumab treatment or within 5 months after the final dose of SG or atezolizumab. Note: Seasonal flu vaccines that do not contain a live virus and locally authorized/approved COVID-19 vaccines are permitted. * Uncontrolled pleural effusion, pericardial effusion, or ascites requiring recurrent drainage procedures (once monthly or more frequently) with the exception of participants with indwelling catheters (e.g., PleurX(R)) who are allowed. * Uncontrolled or symptomatic hypercalcemia (ionized calcium \> 1.5 mmol/L, calcium \> 12 mg/dL or corrected serum calcium \> ULN). * Significant cardiovascular disease (such as New York Heart Association Class II or greater cardiac disease, myocardial infarction, cerebrovascular accident, unstable arrhythmia, or unstable angina) within 3 months prior to initiation of study treatment. * Prior treatment with immune checkpoint blockade therapies, including anti-PD-1, and anti-PD-L1 therapeutic antibodies (for Arm 2 only). * Participants with prior malignancy within the previous 2 years except for locally curable cancers that have been apparently cured such as basal or squamous cell skin cancer, superficial bladder cancer, or carcinoma in situ of the cervix, breast, or low risk Gleason 6 prostate cancer, among others. Participants with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for the study. * History of leptomeningeal disease * Active tuberculosis * Participants with severe uncontrolled intercurrent illness that would limit compliance with study requirements, evaluated by history, physical exam, and chemistry panel.