There are lots of them. I have a japanese recording of Mahler 4th made in the 30s (the first complete recording of this symphony).I don't know where you would get any old orchestral recordings. I have never seen any that were first recorded before the Second World War. I have got some old re-mastered 1950s and 60s recordings on CD, but that's not the same thing.
Acoustic recordings from the beggining of the 20th century usually featured instrumental soloists and singers. Orchestras didn't record a lot as the primitive mics could not capture a wide audio spectrum. For instance, in the orchestral recordings conducted by Ysaye the double basses were replaced with other instruments (don't remember which ones).
Search for Toscanini, he recorded a lot in the 30s.
Sarasate recorded some of his works in 1905 (aprox); others are:
Ysaye, Zimbalist, Joachim, Kreisler, Horowitz, Oistrakh, and many... many conductors.
A lot of operatic stuff was recorded in that time also. Check on Naxos Historical for one of the most touching Turandot you will find ever (with Magda Olivero and Gina Cigna in the cast).