Intermediate
Near-duplicates with shingles and Jaccard
Write shingles(text, k) that lower-cases the text, takes its words (runs of letters and digits) and returns the set of every window of k consecutive words joined by single spaces. A text with fewer than k words gives an empty set.
Write jaccard(a, b) that returns the size of the overlap of two sets divided by the size of their union, as a float. If both sets are empty, return 0.0.
What Check tests
- Shingles are windows of k words
- A short text has no shingles
- Repeated windows count once
- Jaccard is overlap over union
- Identical sets give 1 and empty sets give 0
- The two refund texts are similar but not equal
Stuck? One hint at a time
The solution unlocks after your first Check. Try first: that is where the learning happens.
main.py
Sign in to run your code.
Output
Press Run to see what your program prints.