About the Unique Words Extractor
Before a document goes to a translator, or a glossary gets built for it, somebody has to know which words it actually uses. This tool reduces the text to its distinct vocabulary: every word lowercased, repeats dropped, the remainder sorted alphabetically, with the number of distinct words in the status bar as a rough measure of lexical range.
Words are runs of letters, digits and apostrophes, so contractions survive in one piece and punctuation acts as a separator. Sorting is plain alphabetical order on the lowercased forms, which puts digits ahead of letters and keeps accented forms next to their base letters instead of exiling them to the end.
The result is a list of surface forms, not of concepts. run, runs, running and ran are four entries, since no stemming or lemmatisation is applied, so a long document produces a longer list than a dictionary would. For a terminology check that is the right behaviour, because you want the exact forms a translator will meet. Latin script is what the word pattern covers, so text in Cyrillic, Greek or Japanese is read as separators and will not appear.
For how often each word turns up rather than just which ones do, use the Word Frequency Counter; for whole lines rather than words, Remove Duplicate Lines.
How to use
- Paste the text.
- Copy the sorted list of unique words.
- Use the Frequency Counter if you need counts.
Common questions
- Is it case-insensitive?
- Yes. "Cat" and "cat" are treated as the same word, output in lowercase.
- Are the words sorted?
- Yes, alphabetically.
- How is a word defined?
- A run of letters, digits or apostrophes between separators.