What and where would you recommend to buy souvenirs for kids in New York City, a girl who's 5 and a boy at 3 years? Something not too big and heavy and preferably somewhat related to the city/country, but should not require language skills, as the kids speak another language than English.
The MoMA Design Store has a children's section. You should be able to find something memorable there.
Cool souvenirs at the museums -- American Museum of Natural History would be a natural for kids.
Grand Central Terminal has a really great gift shop there for people of all ages and interests, it's worth checking out.
There were two things that I loved as a kid: toys, and candy :D
I don't have suggestions for what to get specifically, but I can point you to a couple stores where you'll have many choices as you know these children better than I:
For toys, I've always loved FAO Schwarz. It does have historical significance as being the oldest toy store in the U.S. Definitely the coolest one I've ever been to!For candy, there's M&M's World. You'll be in serious chocolate heaven.
This is just me, but I don't necessarily appreciate gifts/souvenirs more just because they have some sort of insignia to a place; just knowing where it's from is cool enough. With that said, though, I'm sure you should be able to find NYC-themed goods at either of these stores.
The best part about these two places is that you'll, as an adult, have fun visiting, too ;)
Post Cards & Magnets! I know it's not a place, but I've found that the trinket-y types of items are often lost on kids.
Puzzles can also work well, especially for the 5 year old.
Hmm, interesting question--for the girl, you might want to pick up an Eloise doll at The Plaza. Eloise is a character in a really famous classic book about a little girl who lives at the hotel (the book was written by Liza Minnelli's Godmother and the character of Eloise is supposedly based on her). Obviously you can't get her the book (although I know it has been translated into a lot of languages and they might have what you're looking for), but you might want to pick up the doll or another small souvenir.
(Eloise boutique at the Plaza)
For the boy, you might want to stop by the New York Yankee Team Store. There are actually quite a few of them and they sell (obviously) Yankee-related gear like t-shirts and hats.
Mentioned in this answer:
I would go to FAO Schwarz just to see it yourself. From experience I still have gifts bought there decades ago, they have the classics, and Steiff animals never break or grow old. As others have mentioned the museum stores are nice and an I love NY tshirt/ hoodie, whatever... can be had in any size anywhere, very cheap.
Ellis Island you can find great statue of liberty mini figurines, foam replicas of the statue's crown, and of course the iconic I love New York T-shirts which is a great gift for all ages.
There are a few places in Times Square that have a huge amount of souvenirs. One I am thinking about in particular that I think is near the McDonalds has a huge kids section!