JavaScript and machine learning: Google shows what’s possible using the web programming language

JavaScript and machine learning: Google shows what’s possible using the web programming language. From head-controlled Pacman to browser-based upload filters — here are some of the best uses of TensorFlow.js.

Building and training machine-learning models using a web-scripting language might seem ambitious, but in 2019 it’s perfectly feasible.

Helping make machine learning possible in the browser is TensorFlow.js, Google’s open-source library for carrying out machine learning using JavaScript. The possibilities opened up by the library were showcased recently with a Google Doodle that generated a fresh Bach-style melody on demand.

In a talk at the Google I/O conference last week Sandeep Gupta, product manager at Google, said TensorFlow.js could be used by developers to create new machine-learning models, as well as to run or retrain pre-trained models.

“It’s built specifically to make it easier for JavaScript developers to build and use machine-learning models within their JavaScript applications,” he said, adding that machine-learning models built using the broader TensorFlow framework can be converted to run using TensorFlow.js.

TensorFlow.js can be used in JavaScript applications running in the browser, on servers inside a Node.js environment, on the desktop using Electron and on mobile browsers on Android and iOS devices. However, it is within the browser that Gupta sees the most possibilities.

“We see a ton of use cases in the browser and it has a lot of advantages because the browser is super interactive, you have easy access to sensors, such as webcams and microphones, which you can then bring into your machine-learning models,” he said.

“Also we use WebGL-based acceleration, so if you have a GPU in your system you can take advantage of that and get really good performance.”

That said, not every developer is won over by TensorFlow.js, with some arguing the library still has significant limitations.

So what exactly is possible using TensorFlow.js? While the framework is still relatively new, only hitting 1.0 this year, Gupta said there has been “really good adoption and usage by the community”, and used his talk to demo some of the most interesting uses of TensorFlow.js.

What’s possible with machine learning and JavaScript?

WeChat Pacman

You might not be crying out for a new way to play Pacman, but TensorFlow.js has made a novel spin on the classic arcade game possible.

Gupta showed a head-controlled version of the game, a JavaScript application running on the WeChat social media messaging platform on a smartphone.

After a quick calibration step, Gupta was able to control Pacman using head gestures, tracked by his phone’s camera, looking left to move left, right to move right, and so on.

“It’s a really fun way of interacting with the device, and the nice thing is that you can do a variety of things using web cams, using text, using speech, and have a very convenient way of sharing these applications without having to install anything,” he said.

Gupta looks to the left to move Pacman around the maze. Image: Google

Uber Manifold

The taxi and delivery company Uber uses machine learning to tackle a wide variety of problems at a very large scale.

Helping it achieve that is Manifold, a browser-based application that Uber uses to visualize and debug their machine-learning models and data pipelines.

“This application runs in the browser and they’re using TensorFlow.js for a lot of numerical computations, so for example distance calculations and visualization, as well as clustering of data,” said Gupta, adding that “because of the WebGL acceleration, they could accelerate these computations more than 100x compared to just natively using JavaScript”.

AirBnB Identity Document Detection

Online property rental service AirBnB uses machine learning in the browser to stop people from inadvertently uploading sensitive information when adding a picture to their profile.

“When a user is trying to upload a profile picture to the AirBnB website, sometimes people accidentally use a driver’s license picture or a passport picture, which may end up containing personal sensitive information,” said Gupta.

“So AirBnB runs a machine-learning model client-side in the browser or on device, so that if you were to choose a picture which may have such sensitive information it will alert you before you upload that picture.”


Clinic.js provides a tool for sys admins and software engineers to profile server-side performance in a Node.js environment.

“This is a node.js-based application which is used for profiling node jobs or node processes and they’re using TensorFlow.js to look for anomalies, or spikes in CPU usage or memory consumption of these node applications,” said Gupta.

Google Creatability

One of the main destinations for showing what is possible using TensorFlow.js is Creatability, Google’s creative labs team’s showcase for experiments using machine learning.

Gupta showed a machine-learning powered demo that allows a person to play a piano keyboard using head gestures.

Other examples of using TensorFlow with JavaScript online include Google’s Gallery page for TensorFlow.js and Magenta.js plug-ins offering machine-learning models for music generation.

Competitors to TensorFlow

TensorFlow isn’t alone in the deep learning field; in fact, there are a number of other companies with machine learning frameworks, including the following.

How do I start using TensorFlow?

The first thing any developer should do is read the TensorFlow Getting Started guide, which includes a TensorFlow Core Tutorial. If you’re new to machine learning, make sure to check out the following guides:

Developers can install TensorFlow on Linux, Mac, and Windows (or even install from source), or check out their various tools from the official TensorFlow GitHub page.

Finally, developers can take advantage of all the TensorFlow guides:

Related Videos:

Related Posts:

Facebook’s chief AI scientist: Deep learning may need a new programming language(Opens in a new browser tab)

Deep learning neural network could reveal why the world works the way it does(Opens in a new browser tab)

I accidently closed my browser, how do I restore the contact form I spent lots of time filling out?(Opens in a new browser tab)

Google has created a maths AI that has already proved 1200 theorems(Opens in a new browser tab)

RobbinHood ransomware takes down Baltimore City government networks(Opens in a new browser tab)

Microsoft is building the full Linux kernel into Windows 10 as a way to pry developers away from their Apple MacBooks(Opens in a new browser tab)

AI and Machine Learning Exploit, Deepfake Videos, Now Harder to Detect(Opens in a new browser tab)

Introduction to JavaScript(Opens in a new browser tab)

Addressing the Biases Plaguing AI Algorithms(Opens in a new browser tab)

The Amazing Ways The Ford Motor Company Uses Artificial Intelligence And Machine Learning(Opens in a new browser tab)

GoogleBot Runs Latest Chrome – This Is Big News(Opens in a new browser tab)

College graduates not learning a programming language that’s vital for top tech jobs.(Opens in a new browser tab)

AI machine learning and deep learning: Everything you need to know(Opens in a new browser tab)

Why Quantum Computing’s Time Is Now(Opens in a new browser tab)

Freedom Mobile hit by data breach,15,000 customers affected(Opens in a new browser tab)

Fake Google reCAPTCHA used to hide Android banking malware(Opens in a new browser tab)

B0r0nt0K Ransomware Wants $75,000 Ransom, Infects Linux Servers(Opens in a new browser tab)

$25 Raspberry Pi add-on gets you started with edge computing AI(Opens in a new browser tab)

Grand theft consciousness: How quantum algorithms will backdoor human-level AI(Opens in a new browser tab)

AI is the future: Microsoft wants to usher it in responsibly(Opens in a new browser tab)

Edmonton doctors, AI scientists team up with drug giant in health software project(Opens in a new browser tab)

Why JavaScript developers are choosing TypeScript(Opens in a new browser tab)

Java, PHP or .NET Which programming languages will earn you the most?(Opens in a new browser tab)

How to open and use the Terminal app on a Mac computer, with a few basic commands(Opens in a new browser tab)


Connected through code, Choose Your Platform!

About the Author: Bernard Aybout

In the land of bytes and bits, a father of three sits, With a heart for tech and coding kits, in IT he never quits. At Magna's door, he took his stance, in Canada's wide expanse, At Karmax Heavy Stamping - Cosma's dance, he gave his career a chance. With a passion deep for teaching code, to the young minds he showed, The path where digital seeds are sowed, in critical thinking mode. But alas, not all was bright and fair, at Magna's lair, oh despair, Harassment, intimidation, a chilling air, made the workplace hard to bear. Management's maze and morale's dip, made our hero's spirit flip, In a demoralizing grip, his well-being began to slip. So he bid adieu to Magna's scene, from the division not so serene, Yet in tech, his interest keen, continues to inspire and convene.