To motivate dynamic time warping, let's look at a classic dynamic programming problem: find the longest common subsequence (LCS) of two strings (Wikipedia). A subsequence is not required to maintain consecutive positions in the original strings, but they must retain their order. Examples:
lcs('cake', 'baker') -> 'ake'
lcs('cake', 'cape') -> 'cae'We can solve this using recursion. We must find the optimal substructure, i.e. decompose the problem into simpler subproblems.
Let x and y be two strings. Let z be the true LCS of x and y.
If the first characters of x and y were the same, then that character must also be the first character of the LCS, z. In other words, if x[0] == y[0], then z[0] must equal x[0] (which equals y[0]). Therefore, append x[0] to lcs(x[1:], y[1:]).
If the first characters of x and y differ, then solve for both lcs(x, y[1:]) and lcs(x[1:], y), and keep the result which is longer.
Here is the recursive solution:
def lcs(x, y):
if x == "" or y == "":
return ""
if x[0] == y[0]:
return x[0] + lcs(x[1:], y[1:])
else:
z1 = lcs(x[1:], y)
z2 = lcs(x, y[1:])
return z1 if len(z1) > len(z2) else z2
Test:
pairs = [
('cake', 'baker'),
('cake', 'cape'),
('catcga', 'gtaccgtca'),
('zxzxzxmnxzmnxmznmzxnzm', 'nmnzxmxzmnzmx'),
('dfkjdjkfdjkjfdkfdkfjd', 'dkfjdjkfjdkjfkdjfkjdkfjdkfj'),
]
for x, y in pairs:
print lcs(x, y)
The time complexity of the above recursive method is $O(2^{n_x+n_y})$. That is slow because we might compute the solution to the same subproblem multiple times.
We can do better through memoization, i.e. storing solutions to previous subproblems in a table.
Here, we create a table where cell (i, j) stores the length lcs(x[:i], y[:j]). When either i or j is equal to zero, i.e. an empty string, we already know that the LCS is the empty string. Therefore, we can initialize the table to be equal to zero in all cells. Then we populate the table from the top left to the bottom right.
def lcs_table(x, y):
nx = len(x)
ny = len(y)
# Initialize a table.
table = [[0 for _ in range(ny+1)] for _ in range(nx+1)]
# Fill the table.
for i in range(1, nx+1):
for j in range(1, ny+1):
if x[i-1] == y[j-1]:
table[i][j] = 1 + table[i-1][j-1]
else:
table[i][j] = max(table[i-1][j], table[i][j-1])
return table
Let's visualize this table:
x = 'cake'
y = 'baker'
table = lcs_table(x, y)
table
xa = ' ' + x
ya = ' ' + y
print ' '.join(ya)
for i, row in enumerate(table):
print xa[i], ' '.join(str(z) for z in row)
Finally, we backtrack, i.e. read the table from the bottom right to the top left:
def lcs(x, y, table, i=None, j=None):
if i is None:
i = len(x)
if j is None:
j = len(y)
if table[i][j] == 0:
return ""
elif x[i-1] == y[j-1]:
return lcs(x, y, table, i-1, j-1) + x[i-1]
elif table[i][j-1] > table[i-1][j]:
return lcs(x, y, table, i, j-1)
else:
return lcs(x, y, table, i-1, j)
for x, y in pairs:
table = lcs_table(x, y)
print lcs(x, y, table)
Table construction has time complexity $O(mn)$, and backtracking is $O(m+n)$. Therefore, the overall running time is $O(mn)$.