Walking past the countless photos of Holocaust survivors and victims at Warsaw’s POLIN Museum of the History of Polish Jews in 2016, New York-native Daniel Patt was haunted by the possibility that he was passing the faces of his own relatives without even knowing it.
For Patt, a 40-year-old software engineer now working for Google, that sort of conundrum presented the potential for a creative solution. And so he set to work creating and developing From Numbers to Names (N2N), an artificial intelligence-driven facial recognition platform that can scan through photos from prewar Europe and the Holocaust, linking them to people living today.
Patt has a personal stake in the project: Three of his four grandparents are Holocaust survivors from Poland, and he wants to help his grandmother find photos of the members of her family murdered by the Nazis
She was just 9 years old when the war started and fled from her hometown of Zamosc eastward with her father and siblings, while her mother — Patt’s great-grandmother — stayed behind. Her mother was shot and killed during the Nazi invasion, and Patt’s great-uncle — his grandmother’s brother — was subsequently killed when he went back to rescue her.
The rest of the family survived and emigrated to New York City after the war.
Currently, N2N’s software — which is free and simple to use — only returns the 10 best potential matches that it can find in the database available to it. Though not yet perfect the nonprofit project has already seen great success: The software has been used to search through hundreds of thousands of photos to identify faces for the United States Holocaust Memorial Museum (USHMM) as well as individual survivors and descendants of survivors — including a number of celebrities.
Patt, who only works on the project on his own time and with his own resources, has now been joined by a growing team of engineers, data scientists and researchers, who are constantly expanding the reach and accuracy of the software.
In addition to the photos and videos currently available to the platform, Patt is working for N2N to gain access to 700,000 more photos from the pre-Holocaust and Holocaust eras.
The Times of Israel interviewed Patt over email on June 26, Holocaust Survivor Day, a newly-minted annual commemoration that celebrates the achievements of Holocaust survivors who have rebuilt their lives and gone on to greatness in response to Hitler’s attempt to eradicate the Jewish people. The following interview has been lightly edited.
The Times of Israel: How did you get the inspiration for Numbers to Names and what is your goal?
Daniel Patt: I started this project after visiting the POLIN Museum of the History of Polish Jews in Warsaw, Poland, in 2016. I couldn’t shake the feeling that I had potentially walked past a photo of a family member without even knowing it. I’m the grandson of Holocaust survivors, all from Poland.
The museum is filled with walls of photos of survivors and victims — but there were only a handful of names to go with them.
One of my grandmothers is now 91. I would like to help her find photos of her immediate family who were murdered by the Nazis. I thought perhaps other Holocaust descendants might want to potentially do the same for their aging grandparents. I’m a software engineer and thought I could build something to help with this “family search” process. This was the original motivation for creating “From Numbers to Names.”
Looking ahead, we’d like for N2N to become a vehicle for Holocaust education, giving students an opportunity to directly contribute to the historical record. Students can use the software to help identify faces and artifacts in photo and video archives and potentially discover new connections between living Holocaust descendants and their ancestors
Can you tell us about some of the successes the platform has had so far?
We reached out to Geddy Lee, from Rush, with a photo we thought was of his mother. He was able to confirm this was indeed a photo of her at the displaced persons camp at Bergen-Belsen.
Geddy was then able to subsequently discover photos of his grandmother, uncles, an aunt and other extended family by browsing the Yad Vashem collection where the initial photo came from.
In addition to our own proactive identification work — some of which has been published on the USHMM website, we’ve also heard about other identifications through various online channels and through organizations such as 3GNY.
We’ve analyzed nearly half a million photos so far, containing around 2 million faces, and there is a backlog of potential identifications we’re manually going through now.
For users coming to the site, how does it work? What is required from them to use it?
For people coming to the site, they can click on “select an image.” They can then select a file from their computer or phone containing a cropped photo of a single face. Then they can click on the “search” button, which will show them 10 photos containing the most similar faces to the one provided by the user. The software works best when searching using photos that are roughly from the same time period (e.g., pre-1960s).
We make no software-based assertions about identifications and leave this judgment to individuals using the site. We simply show results, with similarity scores, and let individuals decide whether the results contain a positive identification.
We’re not all software engineers here, but can you tell us a little bit about how the software works?
The software leverages AI [artificial intelligence] to help Holocaust descendants discover images of their loved ones, and identify the millions of unidentified faces in Holocaust photo and video archives.
We proactively look to identify faces using AI, and also provide a way for people to conduct their own research as well through the numberstonames.org website.
Are you partnered with Holocaust museums or other organizations? How are you trying to grow your database?
We’ve been in contact informally with USHMM. So far we’ve shared a number of identifications which have since been confirmed and published on the USHMM website. We’d like to partner with museums, schools, research institutions, and other organizations which share common goals around Holocaust education, awareness, and so on.
In terms of growing the database, we have begun analyzing videos from the Steven Spielberg Film and Video Archive. We’re also creating a way for verified student and research users to provide annotations and identifications directly on the platform.
You use only your own resources for this project — finances must be something of a hurdle.
The cost of providing quick search results increases as we analyze more photos and now historical videos.
We’re about to release an “offline search” experience also which allows users to get their search results within a day, instead of within seconds as a way of keeping costs down. We have a fiscal sponsorship with A Foundation of Philanthropic Funds as well to help cover costs.
Other than money, what are the biggest challenges facing N2N?
Limited time — we have been developing the project over the course of evenings and weekends over many months. There’s an urgency to this effort as the last remaining survivors pass, and there are many connections that could still be made. We hope that N2N can help build those connections while the survivors are still with us.
No comments:
Post a Comment