The Federal government maintains an online platform that allows the public to weigh in on regulations being considered by Federal departments. When the U.S. Department of Labor (DOL) received over 380,000 comments on a proposed rule, it would have taken considerable time and cost to conduct a traditional analysis. To help expedite the process, IMPAQ augmented the manual tagging of comments with text processing and topic mining algorithms to add a general sentiment dimension to our analysis of the comments.
IMPAQ leveraged Natural Language Processing (NLP) algorithms to provide an objective analysis of general sentiment and quantify social consensus. The text mining procedure was divided into morphological analysis, multivariate analysis (correspondence and cluster analysis), text structural analysis, and concordance analysis.
These steps involved pre-processing the text to remove extraneous words and convert the text into frequencies that are subsequently processed and analyzed to provide meaningful assessments of public sentiments and opinions. Human classifications were used to ensure the computer algorithms were valid.
On this project, the IMPAQ team consisted of Senior Researchers and Analysts with workforce policy experience, as well as Data Analysts with expertise in Natural Language Processing (NLP) techniques and data management.
IMPAQ's notable experience with workforce projects and the U.S. Department of Labor contributed to the success of the project.