Tokenizer tok.py is the basic file that reads from token.csv. filesplitter splits the .csv file into two and tokenizes them