>>316/pol/ack here. I've been watching the series since mid April. Surprisingly I actually have something exactly like what you're looking for.
One oddity that jumped out at me was a direct reference to the France in season one. In season 1, episode 14 (Suited for Success), Rarity makes a bunch of dresses for the cast to wear to the Gala. Each cast member wants to make outrageous changes to their dresses, and Fluttershy explicitly requests details from French fashion be incorporated into her dress.
At the time it made me wonder if the setting would expand to explain how there could be any connection between the ponies and France. While I hadn't seen it yet, I knew that there was that atrocious Equestria Girls spinoff and I wondered if that was connected in any way.
Looking back over that episode, I just now noticed two other oddities. First, the fashion critic remarked that seeing Applejack's dress made him want to eat some Dutch apple pie.
Also of note, Twilight is extremely autistic about the placement of the stars on her dress. The depicted constellations are ones visible in Earth's sky, and she explicitly instructs Rarity about correcting Orion's belt. That's quite odd because Orion is a human figure. In a world without humans, it does not make sense for ponies to recognize and name the same human-like formation of stars.
If you want to go down this rabbit hole, then the above tell us the following about Equestria:
1: Due to the matching constellations, Equestria is Earth. At the very least it is a parallel world that is exceptionally similar to Earth.
2: The recognition of Orion is proof that there was some contact between human culture and equestrian culture. Either the constellations and their names were passed down from generation to generation even after knowledge of mankind faded into obscurity, or equestrian scholars came into contact with human texts and adopted Orion into their constellations. Today they may not know the origin of Orion and may not know that he's human. To them he may very well just be another mythical beast.
3: The french fashion styles and dutch apple pie also suggest that there was some contact between human and equestrian culture. To the ponies, that may very well just be the formal name for that style or that confection. They may have no knowledge of the human nations that were the origin of their styles/deserts.
As I touched upon above, I think that there are two ways that can explain the connection between humanity and equestria. The first, as you suggest, is that the ponies are living a post-human world and that there are select few artifacts that have outlived us.
The second is that the ponies are living in a world parallel to humanity. As much as I'd like to forget Equestrian Girls, it does pretty blatantly support the notion of parallel worlds whose existence is known only to a select few. I don't think that it is beyond reason to suppose that at some time there may have been an equestrian research program to study parallel worlds in search of novel new ideas born from societies who view the world from a different perspective. For example, take
>>318 's post. These observers of parallel worlds may have seen the steam engine and recognized the potential for it to benefit equestrian society. They copied the design but may not have understood how to use what they've copied. That said, some episodes depict the steam engine working as designed and without the aid of puller ponies. Perhaps they eventually figured it out.