Then it is trying to match sensor data arrival time with GPS point time.
When your device time does not exactly match gps time, there will be delays in produced GPX file.
There is a FIX, which worked perfectly for me:
(had to sync device time with GPS time)