INNOVATIVE THINKING
RIGOROUS APPROACHES
REAL WORLD IMPACT

 

Project

Text Processing and Topic Mining of Public Comments on Federal Proposed Regulations

Official Title
DOL MOBIS Economic Impact Analysis, Wage and Hour Division
Dates
06/2017–07/2018
Market
Workforce Development
Client
U.S. Department of Labor - Wage and Hour Division
Methodologies
Natural Languange Processing (NLP)

The Federal government maintains an online platform that allows the public to weigh in on regulations being considered by Federal departments. When the U.S. Department of Labor (DOL) received over 380,000 comments on a proposed rule, it would have taken considerable time and cost to conduct a traditional analysis. To help expedite the process, IMPAQ augmented the manual tagging of comments with text processing and topic mining algorithms to add a general sentiment dimension to our analysis of the comments.

Methodologies

IMPAQ leveraged Natural Language Processing (NLP) algorithms to provide an objective analysis of general sentiment and quantify social consensus. The text mining procedure was divided into morphological analysis, multivariate analysis (correspondence and cluster analysis), text structural analysis, and concordance analysis.

These steps involved pre-processing the text to remove extraneous words and convert the text into frequencies that are subsequently processed and analyzed to provide meaningful assessments of public sentiments and opinions. Human classifications were used to ensure the computer algorithms were valid.

Skills & Experience

On this project, the IMPAQ team consisted of Senior Researchers and Analysts with workforce policy experience, as well as Data Analysts with expertise in Natural Language Processing (NLP) techniques and data management.

IMPAQ's notable experience with workforce projects and the U.S. Department of Labor contributed to the success of the project.