We own two GPS data logger. I see a difference between those two units if i compare those. Is one of those faulty?

No it looks quite the same and exactly fits (what you can expect from GPS accuracy).
Only at the beginning you have a huge error but this for sure only results from bad singal detection or a GPS receiver warm up (not all needed satellites tracked at this point).