Friday, November 4th, 2022

Natural Language Processing

Madeline Drake

GFT (General Fine-Tuning)

Recently, deep nets have demonstrated significant progress with exciting results. Much of this work has been reported in leading media outlets and academic conferences such as ACL and NeurIPS. We have developed a “little” language, GFT, that makes deep nets look like regression. GFT is approachable to a broad audience, and is so easy to use that non-programmers and programmers alike can replicate many of these results.

Although there have been many recent promising results, it is important to set appropriate expectations. One of the points of GFT is to demystify deep nets. No one would suggest that regression-like methods are “magical” or even artificially intelligent. While deep nets can do a lot, there are many classic problems that have been open for decades in AI and for centuries in linguistics that go beyond regression-like methods.

Better Together: Text + Context

We are building embeddings of 200M documents from Semantic Scholar.  Some embeddings capture text (titles and abstracts) and others capture context (properties of other documents such as citations). Embeddings are N by K matrices where N is the number of documents and K are the number of hidden dimensions. Similarity of documents can be computed by cosines of two vectors of length K. Text embeddings are typically based on BERT and context embeddings are based on node2vec encodings of citation graphs.

At the Institute for Experiential AI, we are committed to further exploring and advancing Natural Language Processing research.

Latest Research

Responsible AI

AI Ethics and Responsible AI aim for AI systems that benefit individuals, societies, and the environment. It encompasses all the ethical, legal, and technical aspects of developing and deploying beneficial AI technologies. It includes making sure your AI system does not cause harm, interfere with human agency, discriminate, or waste resources.  AI Ethics and Responsible […]


At the Institute for Experiential AI, we care about people’s health and wellness. We have several core faculty members with AI health expertise who cover AI for healthcare from multiple scales and various heterogeneous data sources. They work closely with scientists, clinicians, and healthcare providers in designing and developing AI algorithms over a wide range […]

Life Sciences

We live in the Golden Age of Life Sciences, driven in part by transformative advances across multiple fronts in our understanding of biological systems and their applications.  However, we are still only at the beginning of this new chapter.  A systematic strategy that includes humans and AI/ML-based approaches can accelerate the “exponentialization” of this value […]

Climate, Sustainability, and Environment

From climate science and water sustainability to urban resilience and disaster preparedness, Artificial Intelligence has shown significant potential to inform and enable transformative action in the areas of climate, sustainability, and the environment. For example, machine learning, graph analytics, agent-based models, and human-AI systems have shown potential with predictive understanding in earth systems science and […]


AI has a lot of potential for cybersecurity applications, such as predicting attacker behavior, learning from existing cyber incidents, and taking proactive defensive measures to protect critical infrastructures. Historically, we proposed techniques for detecting advanced cyberattacks in enterprise networks based on creating semantic representations of network logs and endpoint data, using a range of supervised […]