It's already been mentioned on the show that Danny first joined the Marines before he joined the NYPD; Jamie mentioned that like Sean trying to keep Jack from leaving for college by acting out, he did the same when Danny enlisted. Danny could have either stayed in the Marine Corps Reserve, easily becoming an NCO, while serving in the NYPD and then got called up or volunteered to return to Active Duty after 9/11. Or he could have reenlisted at his old rank, whatever it was.
That's simply the Marine Corps emblem, the Globe, Anchor and Eagle, or more colloquially as the "Hook, Ball and Bird". It's worn in smaller size on the uniform coat lapels and on the folding Overseas Cap. It's worn in larger size on the visored "Saucer Cap" and (in subdued black) on the Drill Instructor "Smokey the Bear" hat. It may look irregular to you because at that angle and it not being fastened to anything, you can see the pins with clutch retainers sticking out of the back
I seem to remember him telling the Army major at Fort Hamilton, when he and Baez were jointly investigating a case with the Army Criminal Investigation Division, mentioning he'd been a sergeant when he did his tours in Fallujah. Since Jamie mentioned being in his early teens when Danny first left for the Marines, Danny presumably served in the Marines before the NYPD and then either continued serving in the Marine Corps Reserve while in the NYPD and then got called up, or decided to reenlist for the Iraq War.