Comment by clauderoux 2 days ago
I implemented many different algorithms for searching and splitting strings using SMID methods as well: https://github.com/naver/tamgu/blob/06aedf2f14895925d7b5a8e2... I used a different algorithm than the ones presented here.
What I do is to build a substring out of the initial strings that is a multiple of 2. If the string I try to search is 9 characters long, then I extract an 8 characters substrings that I transform into an integer over an integer:
Here is the example for 4 characters: //We look for the presence of four characters in a row ``` int32_t cc = search[3]; for (shift = 2; shift >= 0; shift--) { cc <<= 8; cc |= search[shift]; } __m256i firstchar = _mm256_set1_epi16(cc); ```
In this case, I will look for a 4 bytes integers over my sequence: ``` current_bytes = _mm256_cmpeq_epi16(firstchar, current_bytes); q <<= 1; q |= _mm256_movemask_epi8(current_bytes); ```` I'm looking for blocks of 4 characters at a time in my string.
I tried to implement a modified version for LZ77 window search with Zig's generic SIMD a few years ago:
Could you summarize your algorithm? Like a high-level description?
Like algorithm 1 (from the article) checks N possible positions at a time by matching the first and last character; algorithm 2 instead tests the 4 leading characters.