Estimating small time shift between two time series

Question

I have two time series, and i suspect that there is a time shift between them, and i want to estimate this time shift.

This question has been asked before in: Find phase difference between two (inharmonic) waves and find time shift between two similar waveforms but in my case, the time shift is smaller than the resolution of the data. for example the data is available at hourly resolution, and the time shift is only few minutes(see image).

The cause of this is that the datalogger used to measure one of the series has few minutes shift in its time.

Any algorithms out there that can estimate this shift, preferably without using interpolation?

Answer 1

This is quite an interesting problem. Here's an attempt at a partial solution using fourier transforms. This relies on the data being moderately periodic. I'm not sure if it will work with your data (where the derivatives at the endpoints don't seem to match).

import numpy as np

X = np.linspace(0,2*np.pi,30)  #some X values

def yvals(x):
    return np.sin(x)+np.sin(2*x)+np.sin(3*x)

Y1 = yvals(X)
Y2 = yvals(X-0.1)  #shifted y values

#fourier transform both series
FT1 = np.fft.fft(Y1)
FT2 = np.fft.fft(Y2)

#You can show that analyically, a phase shift in the coefficients leads to a 
#multiplicative factor of `exp(-1.j * N * T_d)`

#can't take the 0'th element because that's a division by 0.  Analytically, 
#the division by 0 is OK by L'hopital's<sp?> rule, but computers don't know calculus :)
print np.log(FT2[1:]/FT1[1:])/(-1.j*np.arange(1,len(X)))

A quick inspection of the printed output shows that the frequencies with the most power (N=1,N=2) give reasonable estimates, N=3 does OK too if you look at the absolute value (np.absolute), although I'm at a loss to explain why that would be.

Maybe someone more familiar with the math can take it from here to give a better answer...