Audible to whom? I wonder what proportion of the audience for, say, Beethoven's 6th, can recognise, name and describe the effect of sonata form?
A lay listener probably cannot name sonata form correctly, but they can certainly hear it.
Lets take for example Mozart Symphony 40, 4th Movement:
At the moment the 2nd theme starts, the listener notices that this is a different section of the piece: It's a completely different melody from the 1st theme and also as a much brighter, sweeter mood, which is the result of changing to the relative major key!
In the development section one can clearly hear that it is based on the motive from the first theme. Also, one can hear that this theme is imitiated in the various voices. The increasingly dense contrapuntal texture along with the chromaticism give this sections a more intense and dark mood which is common for development sections.
Once the the recapitulation starts the 1st theme is played again the same way as in the exposition, hence it is easily recognisable as such (which is helped by the fact that it is a quite catchy and memorable theme).
However as the recapulation continues one quickly notices that it is different from the exposition: Instead of moving to a brighter, more cheerful mood, the 2nd theme retains the sad mood of the 1st theme - which is the result of being assimilated to the home key of g minor.
So yes, the structural devices of sonata form are indeed audible to non-musicians. Maybe not every detail, but certainly the most defining aspects.
If they can't, does the symphony suddenly become chaotic?
No. Even if we assume that sonata form can't be heard, the piece still won't sound chaotic. This is due to the fact that sonata form is only one of many aspects of classical music's highly structured nature.
The others are the rules of common-practice tonality: Diatonic scales (major/minor keys), Triadic harmonies (major / minor chords), Harmonic Syntax (cadences, modulation rules), and voice leading rules (resolution of dissonances).
All this things work together to give the music a distinct, structured and non-random sound.
I would argue that simply following the rules of common practice tonality almost inevitably yields highly structured sounding music, which may be the reason why it has survived for so long.