Why are some characters ignored in search queries?

Question
Why can I not search for some special (punctuation) characters? They seem to be ignored in search queries.

Answer

This is because of how the analyzer works. Before an item can be indexed, the analyzer breaks down the text in order to determine the individual words used in it. This analyzer discards whitespace, and most punctuation characters. This means that they will not make it into the index, and cannot be searched. The Tokenization section in the user manual has more information on this.
As whitespace, and most punctuation characters, do not make it into the index, the queries for 'searchterm', 'searchterm/' and 'searchterm ' (with an extra space at the end), will all end up being equivalent.


Question

What if I need to search for keywords that have special characters? Can that be done?

Answer
Yes. If you need to search for keywords that include special characters in Vound's search queries, you may wonder if it is possible to find them. The answer is yes; while special punctuation characters do not make it into the index and cannot be searched using standard keyword searching methods such as the search box or keyword lists, there is a solution.


Vound has another database that holds all of the text, including all special characters that are present in the content. You can locate that database by using a regular expression search. To use regular expressions in Vound's search feature, you can set them up within the Content Analysis facet. Regular expressions provide a powerful tool for searching as they allow you to use complex patterns to search for specific strings of text, including those that contain special characters. By using regular expressions, you can locate keywords or phrases that contain special characters which are otherwise excluded from the index.

While regular expressions can be a little bit more complex than standard keyword searches, they provide users with a versatile way of searching for content that contains special characters. Regular expressions come with several powerful features that can help you locate specific strings of text and refine your searches further. Detailed information on using regular expressions can be found in the Content Analysis section of the user manual.


More information regarding searching and using regular expressions in Intella and Connect can be found in this webinar at the link below.

https://community.vound-software.com/topic/984-talking-tech-regex-searches-a-better-understanding-of-searching-in-intella/ 


Updated April 2023

Tags