Thank you, Chairperson, for giving me the opportunity to speak at this session.
My name is Saar Abramovich, Patent Examiner; I am taking part in a team at the Israeli Patents and Trademarks Office, which examines the use of search engines based on AI technology, designed to find prior art documents during the process of patents examination, and I wish to share some insights from ILPO's experience in that field.
With the exponential growth of scientific and technical information, the increasing number of patent applications every year, and the use of AI tools in the process of research and development, the ILPO believes AI-based search engines has potential to improve the efficiency and quality of patent searches.
During the last two years the ILPO has tested 4 state of the art systems through a series of comparative studies.
The tests were designed to compare the AI systems performance to human examiners performance, and included two complementary questions:
- Does the system find the same prior art pieces found by a human examiner, and
- Does it also find prior art not found by the examiner?
The system with the best results was Similari, of an Israeli startup company by the same name. Similari was successful in finding a third of the prior art documents cited in search reports conducted by examiners without the aid of AI system; these prior art documents were included amongst the first 100 results of the system.
Similari's AI search engine was also occasionally successful in adding relevant publications to those found by examiners.
Based on these results, the office currently uses the AI system regularly as starting point for searches in the fields of computers, medical-instruments, physics, electronics and mechanical engineering, both as international search authority and as a national registrar.
The examiners are instructed to go through the first 50 publications Similari finds, and complement the search with a "traditional " search where needed.
At this stage, the system is not making any decisions regarding the patentability of the applications. All decisions are made by the examiners.
The search strategy document for each application, specifically points out the use of an AI search engine; we consider full transparency in that aspect to be obligatory.
During the process, we came to learn that there are many methods of evaluating the performance of AI based search engines.
Defining standard tests for the evaluation of AI-based search engines, similar to those existing in other fields such as image recognition, will allow the IP national and international offices, as well as other stakeholders, to compare results of their testing of such systems, by an agreed-upon scale; to direct the industry, and to help in development and improvement of such systems.
Such standard may establish data-sets, test-sets, methods of grading each search result, and methods of presenting an overall measure of the engine quality.
WIPO can be good facilitator for such discussion, which requires IP offices, subject matter experts and industry.
The Israeli Patent Office will be happy to contribute to such discussion as well as exchange knowledge and information with the other IP offices on the broader subject of using AI in the of process of IP registration through WIPO and other platforms.
Thank you Chairperson.