School of Public Policy Study Shows Artificial Intelligence Can Beat Human Experts in Discovering Attitudes on Electric Vehicle Charging

Posted January 22, 2021

A research team led by School of Public Policy Assistant Professor Omar Asensio used artificial intelligence to analyze consumer data from electric vehicle (EV) charging station reviews, besting people who analyzed and annotated the same data. The algorithms also detected social inequities in charging station availability across the United States, according to their research published Jan. 22 in the journal Patterns.

“By deploying data strategies that can generate real-time insights at different scales of analysis, we’ve demonstrated that artificial intelligence can provide inexpensive, highly accurate insights for industry and policymakers seeking to build out the electric vehicle charging infrastructure,” Asensio said. “This is an important goal given the potential climate-change benefits of emission-free vehicles and related policies.”

Asensio co-authored the Patterns paper with student researchers Daniel Marchetto, a Ph.D. student in Public Policy; Susie Ha, a dual Ph.D. student in Computational Science and Engineering and Civil and Environmental Engineering; and Sameer Dharur, a master’s student in Computer Science.

The work builds on earlier efforts by Asensio and his Georgia Tech students using deep learning models to assess consumer sentiment from unstructured text reviews of charging stations submitted to popular smartphone apps for EV drivers.

That research showed promise using a convolutional neural network to detect consumer sentiment in near real-time. The technique rivaled human performance in classifying sentiment. However, being able to detect the reasons for consumer positivity or dismay remained an open problem.

In this new research, however, Asensio’s team shows that transformer-based deep learning, a different machine learning technique, was able to discover topics of consumer discussion automatically. In some cases, the models did better than expert human annotators in labeling the conversations.

For instance, consider the review, “Thanks very busy dealership but happy to allow use of qcdc.” The model was able to identify labels accurately representing the reasons for the review: functionality, availability, and “dealership.” Asensio’s team used the latter label to describe comments concerning specific dealerships and users’ associated charging experiences.

People did not do quite as well as the transformer-based BERT (bidirectional encoder representations from transformers) model in applying the correct labels, according to the research. The BERT model applied the correct labels to reviews 91.6% of the time. That model and another used by researchers, XLNet, in some cases performed 3- to 5-percentage points better than human experts.

“We are at a new milestone of research when AI beats humans, especially in domains where cheap crowd-sourced labels are difficult and costly to get,” Asensio said.

Analysis Reveals Inequities

In addition to the classification advances, Asensio’s team spotted a pattern of station availability issues from existing EV users. The study merged the AI model predictions with location features to evaluate possible differences by region. The study found evidence that consumers report station availability issues most frequently in smaller cities, particularly in the West, Midwest, and Hawaii.

The fact that station availability issues tend to dominate in smaller communities with 10,000 to 50,000 residents suggests that additional mechanisms are needed to broaden access to sustainable charging infrastructure, Asensio said.

These findings are notable because the United States lacks federal innovation policies that could address these potential gaps, which Asensio said result from a decentralized model of growth.

“The latest theory and evidence suggest that it may be more effective to subsidize EV station deployment, rather than subsidize EV car sales. However, until the advent of these AI-driven tools, we didn’t have a practical way to discover large-scale behavioral issues,” Asensio said.

The team is now rapidly accelerating efforts to better understand equity issues in the availability of station infrastructure for hard-to-reach communities, especially considering the current rapid global investment in the area.

Advances Could Fuel Policy Innovation

The advances demonstrated in Asensio’s paper buttress the case that machine learning tools could uniquely address the need for real-time consumer intelligence related to electric mobility.

Such improvements are significant because of the unique role charging infrastructure plays in supporting the sale of electric vehicles, an expanding market seen as a critical tool to help reduce greenhouse gas emissions and combat climate change.

The notable advances also have broader implications for public policy work in general, according to Asensio and his team.

“The extent of this improvement could significantly accelerate automated research evaluation using large-scale consumer data for performance assessment and regional policy analysis in other domains,” the team wrote in the paper.

The article, “Topic Classification of Electric Vehicle Consumer Experiences with Transformer-Based Deep Learning,” is available online at https://doi.org/10.1016/j.patter.2020.100195.

The School of Public Policy is a unit of the Ivan Allen College of Liberal Arts.