Following a long string of victories for computers in other games — chess in 1997, go in 2016 and Texas hold’em poker in 2019 — a GPU-powered AI has now beaten some of the world’s most competitive word nerds at crossword puzzles.
Dr.Fill, the crossword puzzle-playing AI created by entrepreneur, AI researcher and former research professor Matt Ginsberg, scored higher than any humans last month at the American Crossword Puzzle Tournament.
Dr.Fill’s performance against more than 1 300 crossword enthusiasts comes after a decade of playing alongside humans through the annual tournament.
Dr.Fill’s edge was a sophisticated neural network developed by UC Berkeley’s Natural Language Processing team — trained in just days on an NVidia DGX-1 system and deployed on a PC equipped with a pair of NVidia GeForce RTX 2080 Ti GPUs.
“Crossword fills require you to make these creative multi-hop lateral connections with language,” says Professor Dan Klein, who leads the natural language processing team. “I thought it would be a good test to see how the technology we’ve created in this field would handle that kind of creative language use.”
Given that unstructured nature, it’s amazing that a computer can compete at all. And to be sure, Dr.Fill still isn’t necessarily the best, and that’s not only because the American Crossword Puzzle Tournament’s official championship is reserved only for humans.
The contest’s organizer, New York Times Puzzle Editor Will Shortz, pointed out that Dr.Fill’s biggest advantage is speed: it can fill in answers in an instant that humans have to type out. Judged solely by accuracy, however, Dr.Fill still isn’t the best, making three errors during the contest, worse than several human contestants.
Nevertheless, Dr.Fill’s performance in a challenge that, unlike more structured games such as chess or go, rely so heavily on real-world knowledge and wordplay is remarkable, Shortz concedes.
“It’s just amazing they have programmed a computer to solve crosswords — especially some of the tricky hard ones.”