I could find no record of the births of Manfred, Chico, or Harpo. Although we know from Manfred’s death record that he was born about January 1886, and we know from other (later) records that Chico was born 22 Mar 1887, and Harpo was born 23 Nov 1888.
The three records below, for Groucho, Gummo, and Zeppo, are from the New York City Department of Records & Information Services, who finally put full color scans up at their site in March 2022.
But here’s Groucho from 2 Oct 1890. Born at 239 East 114th St.

And Gummo (Milton, here as Moses) from 21 May 1893. Born at 137 East 119th St.

And here’s Zeppo’s from 26 Feb 1901. Born at 179 East 93rd St.

And the sort-of Marx sister (cousin, really), Pauline from 10 Jan 1884. Whose actual father has always been a mystery. Her mother was Minnie’s sister Hannah (here, for some reason, as “Pauline”). Hannah was married to Max Levender 26 Jun 1881. But he disappeared at some point. Harpo says Pauline had “been adopted as one of us” and was with the family on the 1900 census.
The thought is that Pauline may have been illegitimate. But this points to a conception date of about April 1883. And Max is in the Manhattan directory of 1883 (but not 1884).

Here’s Maxine’s Cook County birth certificate. Note the middle name. although the first and middle names seem to be in different handwriting. Gummo was her godfather.

Going further back in time we get to Frenchie’s birth certificate.

(Simon) Samuel Marx, b. 23 Oct 1859: Mertzwiller, Departement du Bas-Rhin, Alsace, France. To Hannah Haennchen Adele Levi, age 20, and Simon Marx, age 29.

And even further is Frenchie’s father Simon. Born on 2 Jul 1830, in Hochfelden, Bas-Rhin, France, to Babete Weil, age 26, and Simon Kronscher Marx, age 26.

Minnie was born on 9 Nov 1864, in Dornum, Lower Saxony, Germany, to Fanny Sophie Salomons, age 33, and Louis Levy “Lafe” Schoenberg, age 34. But I couldn’t find a primary source document.
A good source of Marx genealogical data can be found here:
judaisme.sdv.fr/perso/marxbr/geneal.htm
And here, just for near-Marxian completion, is Margaret Dumont’s birth record. As Daisy Juliette Baker. Born at 10am on 20 Oct 1882 at 157 Washington St in Brooklyn.
