Process of converting a sequence of characters into a sequence of tokens.
I'm trying to find out if there is a known algorithm that can detect the "key concept" of a sentence. …
algorithm nlp semantics lexical-analysisint main() { i/*nt*/a = 10; return 0; } If I have the above code and I want to count the tokens, will …
c lexical-analysisMy little project is a lexical analysis program in which i have to take every word found in an arbitrary .…
java lexical-analysis lookup-tablesI am looking for a simple compiler that compiles a simple language, I need it to write a paper about …
parsing code-generation lexical-analysis compiler-constructionI'm using Flex and Bison for a parser generator, but having problems with the start states in my scanner. I'm …
parsing bison lex lexical-analysis flex-lexerI am having a big problem to write a regexp that will trim all the whitespace in my input. I …
regex compiler-construction flex-lexer lex lexical-analysisI find Ruby's each function a bit confusing. If I have a line of text, an each loop will give …
ruby parsing delimiter lexical-analysis tab-delimitedI'm looking to speed along my discovery process here quite a bit, as this is my first venture into the …
python transform lexical-analysisI realize this is a broad topic, but I'm looking for a good primer on parsing meaning from text, ideally …
python parsing nlp lexical-analysisBesides not closing a comment /*..., what constitutes a lexical error in C?
c compiler-errors lexical-analysis