for Dr. Naeemul Hassan 's CSCI 444 - Data Visualization
Keep reading or
Explore visualizationsFor the past decade, an increasing number of foreigners have been coming to the US to work and reside. PERM is the process for obtaining a permanent labor certification, the first step of the green card process for foreign nationals seeking permanent residence through their employment.
A permanent labor certification issued by the Department of Labor (DOL) allows an employer to hire a foreign worker to work permanently in the United States. In most instances, before the U.S. employer can submit an immigration petition to the Department of Homeland Security's U.S. Citizenship and Immigration Services (USCIS), the employer must obtain a certified labor certification application from the DOL's Employment and Training Administration (ETA). The DOL must certify to the USCIS that there are not sufficient U.S. workers able, willing, qualified and available to accept the job opportunity in the area of intended employment and that employment of the foreign worker will not adversely affect the wages and working conditions of similarly employed U.S. workers.
Which country, state have the highest acceptance rate?
What is the average processing time for each country? average salary for each job?
What are the jobs with the most certified cases in each state?
What is the most demanded job?
What are the job trends over the years?
Will we be able to "predict" a PERM application outcome based on our observation?
We are going to answer the above questions by providing visualizations regarding each country ( world map), state ( U.S. map), occupation and occupation group ( stream graph & tree map), and admission class / visa types ( brushable bar chart).
The dataset we're using is available here. We have intensively processed the dataset by filtering out unnecessary attributes, normalizing and deriving multiple attributes for our visualizations. Below is a list describing the main columns in our final dataset.
Original Attribute - Unchanged
Once a person applied for a permanent labor certification, their application can be decided as: "Certified", "Certified-Expired", "Denied" and "Withdrawn." We mainly focused on Certified and Denied cases.
Original Attribute - Unchanged
Info regarding the visa classification that the person applying were admitted into the U.S. on their last arrival.
Original Attribute - Normalized
The country in which the person was born and has not renounced or lost citizenship. We have process to make all values uppercase in our dataset to prevent duplicates.
Derived Attribute
Info regarding the duration from when an application is received to when the application status is decided. We simply calculated this value from "Received Date" and "Decision Date" attribues.
Original Attribute - Normalized
The state where the offered job is located.
Original Attribute - Normalized
The 2000 Standard Occupational Classification (SOC) System was developed in response to a growing need for a universal occupational classification system. It classifies workers at four levels of aggregation: 1) major group; 2) minor group; 3) broad occupation; and 4) detailed occupation. Each item in this hierarchy is designated by a six-digit code - an SOC Code.
Original Attribute - Partially Normalized
All together, there are 821 detailed occupations. Even though this data is presented in the dataset, there are many inconsistencies in the value. Since there are also inconsistencies in the SOC Code, we had to manually check each job with this file from Bureau of Labor Statistics, which is why the values in this attribute are "partially normalized". We were able to determined the major groups of all occupations. However, we marked the minor groups of those that are not normalized as "N/A".
Derived Attribute
All occupations are clustered into 23 major groups. Since the dataset doesn't include this info, we have used the SOC Code to obtain it. The major group information is presented by the first two digits of the SOC Code. Major groups of jobs that are missing "job title - detailed occupation attribute" are marked as "N/A".
Derived Attribute
Within all major groups are 96 minor groups. We also used the SOC Code and Python scripting to obtain this data. The third digit of the SOC Code represents the minor groups and all minor groups SOC Code ends with 000.
Derived Attribute
Info regarding the offered annual wage of an application. We simply calculated this attribute from the "wage offered" amount and "wage offered unit of pay". During this process, we made assumptions regarding the number of weeks (52) and hours (2080) of work in a year to calculate the yearly amount based off unit of pay.
This heat world map shows data regarding the case status of all applications from each country over the past 6 years (2011 - 2016). The map provides 2 color scale: one is for the percentage of certified cases in each country, and one is for the total number of certified cases. Hover over each country for details.
Select an option from the radio button to change color scale.
Hover over a country to view details regarding case status.
As we can see, most countries' certified rate is around 30-50%. India is the country with the most certifed cases (as well as total cases). China comes second, but still almost 90,000 total cases less than India. This might seem unrealistic. However, it is a reasonable number as we will later on discover that "Computer and Mathematical Occupations" account for the most of the certified cases of all major occupation groups, and India is known for having many people who are good in this field.
The chart below is currently showing the average processing time in days for each country, where only certified cases were taken into consideration. However, different options are also available: Total certified cases and average processing time for all cases. With this visualization, you can compare the average processing time/total certified cases between countries as the values are in descending order.
Select an option in the dropdown box to view the chart in different data.
Drag the box in the mini chart,
scroll your mouse or
click anywhere in the mini chart to see different areas of the total bar chart.
Resize the box in the mini chart to adjust your view size.
The x axis will update automatically based on the viewable items.
My fellow Americans, we are and always will be a nation of immigrants. We were strangers once, too.
Let's now take a look at the overview of PERM applications where the jobs are located in different U.S. States.
This heat map behaves just as the world map. It also provides 2 different scales: one for the percentage of certified cases in each state, and another for the total number of certified cases.
Select an option from the radio button to change color scale.
Hover over each state for information in corresponding to the scale.
Click on a state to enter a modal which contains a bar chart of case status corresponding
to the scale.
While in the modal,
compare the total number of each case status by selecting another state.
By looking at the map showing certified percentage, it's clear that Mississippi has the lowest certified rate at just above 36%, while state like New Hampshire, Oregon and Missouri top the category, all of which have over 50% certified rate. Most states sit at around 48-50% certified rate.
Regarding the total number certified cases, California accounts for a very large amount, at about 43,000 certified cases. Texas comes second at about 23,000 certified cases and the state of New York is third with over 14,000 cases. This is again, very understandable when we take into account the fact that California is the technology hub, "Computer and Mathematical Occupations" are the majority of all major occupation groups and India tops all countries with most certified cases.
Here, we use another brushable bar chart to rank the top major occupation groups with in terms of certified cases in each state. While most states have "Computer and Mathematical" as their top major group, Mississippi, the state with the lowest certified rate, have "Production", "Education, Training and Library" and "Healthcare Practitioners and Technical" Occupations as its top 3.
Select a state.
Adjust the size of the brush to view all major occupation groups. The chart shows top 7
groups by default.
In order to understand the data regarding occupations, a stream graph is used to show the trend over 6 years of the total
number of certified cases for each major groups. There are 22 stacks, representing each of the 22
major groups with
22 distinct colors. The order of the stacks is that the major groups with the smallest total
of certified cases (over all years) will be on top, while groups with the largest total of certified
cases (over all years) will be in the bottom.
From the graph, there are
three clear observations:
- The total number of certified cases is on the rise over the years.
- The major group that has the most certified cases is always "Computer and Mathematical Occupations".
- Since 2012, there is a period of a few consecutive months every year (From October - March)
where no applications are processed.
Hover over a stack to view the number of certified cases of a group in a certain month/year.
Move your mouse to see the changes in certified cases in different time periods.
Another visualization we use to understand occupations is a treemap. The treemap provides a hierarchical view of our occupation data, showing the major groups, minor groups and all detailed occupations within each group. With each treemap, we can also easily spot a pattern. For example, within major group "Business and Financial Operations Occupations", minor group "Financial Specialists" accounts for half of the total certified cases.
One interesting observation from the "Total Annual Wage" treemap is that: even though "Architecture and Engineering Occupations", "Management Occupations" and "Business & Financial Occupations" all have higher numbers of certified cases, "Healthcare Practitioners and Technical Occupations" account for a much larger amount of annual wage (double the amount of Management Occupations which come right after it).
Select an option from the radio button to change the map data.
Use the stream graph legend as reference to look up for major groups. Don't use it for minor
groups and detailed occupations.
Click on the rectangulars to enter the next hierarchical level. The 3 levels are Major Groups
- Minor Groups - Detailed Occupations.
Click on the top rectangular to exit to the previous hierarchical level.
Hover over a rectangular does 2 things:
- Show you a sneak peak of what items are inside the next
hierarchical level.
- Provide you a tooltip that tells you the item's name,
value, and what hierarchical level you're in.
In the last section, we will explore the data regarding "Class of Admission", or type of visa. We will only be using our
familiar brushable bar chart. The interactivities of the chart stay the same, while there are 4 different
options: Certified Percentage, Certified Total, Denied Percentage and Denied Total.
H-1B is clearly the top of the admission class with almost 140,000 people who got certified. Additionally,
most visa types have a certified rate of around 45-50%.
One of our main question when we first decided to do this project is that whether or not we can use these dataset and visualizations
to predict a PERM application outcome.
Provided information regarding country of citizenship, job, offered salary, time of application
and job location, we can query our dataset to get the matching records, and from there determine
what the certified rate is. This can potentially help users figure out a way to optimize their permanent
visa application for the highest chance to be certified.