Kimia Lab News

The Pathology Moonshot – We need 5 Million WSIs to Train a Foundation Model

Posted: 2023/07/28 by Hamid Tizhoosh

Foundation Models

The term foundation model (FM) in AI generally refers to large-scale (very deep) language models that can be utilized as a pre-trained and fine-tuned backbone (foundation) for many different applications. FMs are typically pre-trained on massive amounts of data to capture the intrinsic patterns, relationships, and representations of natural language (and increasingly images). FMs can be fine-tuned – after they have been pre-trained – for specific downstream tasks to deliver high-performance results on a wide range of applications.

Figure 1 uses coarse lengths to conceptually visualize ratios of DenseNet with 7 million parameters which is not a foundation model, and the BERT model with 350 million parameters, as well as chatGPT with 1.5 billion.

FMs are exhibiting exceptional language understanding and generation capabilities. Some of the well-known FMs include OpenAI’s GPT (Generative Pre-trained Transformer) series (such as GPT-3), Google’s BERT (Bidirectional Encoder Representations from Transformers), and Facebook’s RoBERTa (A Robustly Optimized BERT Pretraining Approach), among others.

Most FMs use the transformer architectures, which allows them to capture long-range dependencies in text through connectionist correlation analysis, making them highly effective for a wide variety of NLP tasks, including, translation, question-answering, and summarization. FMs can drastically reduce the need for extensive task-specific training data and computational resources if we employ transfer learning to exploit the knowledge learned, particularly through one-shot and few-shot learning.

FMs are becoming indispensable tools in dealing with text and other data modalities. However, there are also many ethical concerns attached to their training and usage. From impacting the employment landscape and environmental concerns (due to massive usage of computing resources) to bias and fairness, FMs may invoke controversial debates. For the medical field the questions of patient consent and privacy are crucial as well as the danger of fabrication and manipulation. 

What do we need

Design and training FMs both pre-training and fine-tuning. During the massive pre-training, FMs learn from huge amounts of unlabeled data. The subsequent fine-tuning helps the model to adjust to specific downstream tasks using labeled data. For any conversational model, a large part of the FM must be based on the transformer architecture with layers of self-attention for discovering and exploiting long-range dependencies in the text. Without a transformer the true potentials of FMs for histopathology cannot be exploited.

One needs an insanely large and diverse corpus of data to train FMs. This commonly includes – for general purpose FMs – books, articles, websites, and social media posts, among others. Of course for histopathology, most of these sources may either be infeasible (e.g., copyright issues for medical books) or are not reliable (e.g., website and social media).

It is common knowledge that training a foundation model needs massive computational resources such as powerful GPUs or TPUs.

Creating input-output pairs for pre-training would be a required pre-processing, a task that is a bit harder for histopathology than for general-purpose FMs. Whereas fine-tuning data after pre-training may be easy for general-purpose FMs (e.g., through crowd-sourcing), this will be a real challenge in histopathology. The number of pathologists is low and the ones in clinical practice are in high-demand. Hospitals won’t be able to spare pathologists’ time for labeling data for FM fine-tunings. The model validation in histopathology cannot be performed with usual metrics; it has to be done live in clinical practice as a simultaneous research project. It would take months to properly validate a FM.

CLIP: The Role Model?

CLIP stands for “Contrastive Language-Image Pre-training”, a FM developed by OpenAI that bridges natural language and digital images by learning “joint representations” of images and their corresponding textual descriptions. Based on availability of a large dataset containing images and their associated textual descriptions, CLIP learns to map similar images and their descriptions in a close proximity in the feature space while, at the same time, pushing dissimilar images and descriptions farther apart.

CLIP can perform “Image-to-Text Matching” when for any given image it predicts the most relevant description from a set of candidate descriptions. CLIP is bidirectional and can perform “Text-to-Image Matching” as well; Given a description, CLIP predicts the most relevant image from a set of candidate images. Various vision-related tasks can be performed with few-shot learning, where it can recognize and generate descriptions for images without extensive task-specific training.

CLIP has demonstrated impressive capabilities across a range of tasks, such as image classification, object detection, and even generating images from textual descriptions. Its ability to generalize across diverse datasets and tasks has made it a versatile and powerful tool in the field of AI research and applications. However, CLIP does not offer context-based conversations like chatGBPT.

Requirements for a FM in Histopathology

GPT-3 has 175 billion weights (300-500 petaFLOPS) and has used 4096 GPUs to be trained on hundreds of Terabytes of text data. And CLIP, to come closer to histopathology, used 400,000,000 image-caption pairs for its training. These numbers should guide us in the right direction of what data volume may be needed for us in histopathology to train a FM.

In computer science we talk about “Garbage In, Garbage Out”, a fundamental principle, short GIGO, that emphasizes the importance of data quality in producing meaningful and accurate output from  algorithms and models. GIGO basically states that if we provide incorrect or low-quality data as input to a computer program or system, the resulting output will also be incorrect or of low quality. In other words, the quality of the output is entirely dependent on the quality of the input. GIGO means that even the most sophisticated AI models will not produce reliable results if trained with low-quality data (inaccurate or irrelevant). 

The quality of general (non-medical) text and images may be easily verifiable. The quality verification of the photo of a dog and its description “a dog plays in the park” does not pose an insurmountable challenge. However, in medicine both images and their descriptions are much more complex. The computerized quality control of medical data (meaning, association and relevance) is practically an unsolved problem that needs expert intervention. CLIP and other FMs may be able to get their data from the Internet but this is not a sane approach in medicine. Scraping histopathology data from the Internet is a clumsy approach that may harbor serious dangers for downstream tasks. We should avoid working with Internet data for medicine, even for research.

In histopathology we deal with whole slide images (WSIs). These are gigapixel color images that can easily be 100,000 by 100,000 pixels. Assuming every WSI has only one relevant region (i.e., abnormality or malignancy) that matches the diagnostic report or the molecular data, and assuming that “patching” (Figure 2) can serve as augmentation (many patches for the same text/caption), then we may be able to estimate how many WSIs we need to properly train a foundation model for histopathology. The two stage k-means clustering proposed by the search engine Yottixel provides a “mosaic” of an average of 80 patches, each approximately 1000 by 1000 pixels.

Considering these facts we should also recall that the proper training of CLIP took 400,000,000 image-caption pairs. That means to do something comparable in histopathology we would need 5,000,000 WSIs and their corresponding reports. This must be supplemented with corresponding lab data, radiology, genomics (DNA, RNA), social determinants and any other data type attached/related to those WSIs. It will be a monumental data management project to keep WSIs in the same place but perhaps pull the other data during the training.

To this date, the largest public WSI dataset (namely TCGA) has barely 40,000 mixed WSIs (frozen and paraffin-embedded) with sparse textual descriptions. Scraping data from PubMed and other heterogeneous datasets to find image-text pairs relevant to histopathology will be working at the event horizon of the GIGO blackhole. The field of medicine needs high-quality clinical data to train FMs for pathology, radiology or any other subfield. That takes time. Serious researchers should exercise patience and put things together for the histopathology moonshot: train a foundation model with high-quality data from one or more hospitals.

Author: H.R. Tizhoosh

Ethics in the AI Publishing World — Is Repackaging Ideas Plagiarism?

Posted: 2023/07/28 by Hamid Tizhoosh

As digital technologies advanced, simple text comparisons became capable of easily detecting plagiarized text, even when it was deceptively modified. In response, thieves shifted their focus to plagiarizing ideas. Oxford University defines plagiarism for its students as “Presenting work or ideas from another source as your own, with or without consent of the original author, by incorporating it into your work without full acknowledgement” [3]. However, the concept of “full acknowledgement” can be open to interpretation.

Who plagiarizes ideas is required to formulate the idea using his own words, displaying a level of intelligence to disguise the theft. By omitting citations to the work of their victim, the thieves claim the idea as their own. Although these cases are more complex, they can still be detected through time stamps and track records, allowing the original creator to be identified.

In today’s AI-driven era, researchers face immense pressure to publish rapidly and frequently. The phrase “Publish or Perish” has evolved into “Publish Fast or Perish Soon.” This pressure has driven some researchers to engage in unethical practices such as data fabrication and the omission of crucial details in order to gain recognition [4, 5]. Both cheaters and thieves operate in an environment with lowered ethical standards, driven by the intense pressure to succeed.

The Era of RePackaging?

In recent years, plagiarism detection technology has significantly improved [6]. It has become more challenging to simply steal intellectual property, leading many clever plagiarizers, especially in lucrative fields like data science and AI, to adopt a different approach known as “repackaging.” Repackaging can be considered a more sophisticated form of plagiarism.

To successfully repackage an idea, algorithm, or deep model, one must have expertise in the field. One needs to possess knowledge of the terminology and be familiar with “uncommon” methods that can be used to create the illusion of novelty during the repackaging process. Essentially, the core ideas or original algorithm or processing chain are taken, and new components that are relatively uncommon are added. This is then combined with a level of ambiguity and a multitude of results to create a repackaged work. To further enhance its appeal, the repackaged work is given a new name and, if the individual is particularly savvy, a flashy and captivating acronym. In this way, a paper with a “new” idea that appears superior to others is created, allowing the repackager to claim ownership over it.

However, one crucial aspect is that the original work must be cited, preventing accusations of plagiarism from both humans and plagiarism-detection software. The repackagers can defend themselves by saying, “I have cited your work, haven’t I?”

Prerequisites for Repackaging Ideas

To successfully execute repackaging, several other factors come into play. Firstly, it is important to note that repackaging typically occurs in a downward direction and not in an upward direction. This means that individuals with lesser-known or unknown affiliations cannot effectively repackage the ideas of those affiliated with prestigious institutions such as Ivy League schools or large corporations with extensive research activities. These institutions and individuals are often powerful, organized, and capable of taking legal action against cases of upward repackaging. Publishers are more likely to believe the claim that the average individual has stolen ideas from a more accomplished individual, making the chances of success for upward repackaging highly unlikely.

Downward repackaging, on the other hand, involves a researcher or team of researchers with an excellent affiliation attempting to steal ideas from individuals associated with less prestigious institutions. In both cases, the terms “excellence” and “mediocrity” refer to the institutions rather than the individuals themselves. The individual from a lower-ranked institution who engages in repackaging is likely to fail as their assertions are not generally believed. Conversely, the individual from a higher-ranked institution who repackages ideas from a lower-ranked institution is more likely to succeed as their claims are more readily accepted.

It is common knowledge that the ranking and status of one’s affiliation can significantly impact their career. When it comes to publications, institutional bias or favouritism is a recognized but challenging phenomenon to detect and objectively measure [7, 8]. Bias within the publishing industry can take various forms [9]. Elite schools with established fame and reputation often benefit from favouritism and biased evaluations [10, 11, 12].

The motivation behind repackaging ideas from less prestigious sources is not simply a matter of intelligence or competence. It involves a combination of factors. Firstly, there are overambitious researchers, both students and professors, who are eager to publish groundbreaking works in the field of AI. These ambitious authors may not have a strong understanding of the nuances of academic integrity. However, it is expected that a seasoned advisor with a high level of academic integrity would guide and teach them the ethical principles of conducting research. The presence of a young and inexperienced advisor, who is perhaps more distinctly driven to publish, can contribute to a lax approach to academic honesty. In such cases, the advisor may not effectively enforce ethical standards and may be more willing to compromise on integrity for the sake of achieving ambitious research goals. This combination of an overambitious student and an inexperienced advisor within a prestigious institution may create an environment conducive to repackaging ideas. This scenario is often observed because researchers at elite schools and institutions face heightened pressure to be both creative and productive. The pervasive “publish or perish” culture amplifies the pressure on these individuals, making them more susceptible to ethical lapses as they strive to meet the high expectations placed upon them.

However, despite these concerns, it is important to note that dishonesty, even in complex forms such as repackaging, will eventually be identified by the peer review system. Senior scientists often acknowledge this fact. Nevertheless, the peer review system is not without its flaws. Improperly selected reviewers may fail to detect insidiously repackaged ideas, and biased journals may prioritize articles from higher-ranked institutions in order to enhance their own reputation. It can be seen as advantageous for an editor when a majority of articles in their journal originate from prestigious institutions. This issue may become even more significant when dealing with editors who receive a salary for their position. While professors generally take on editorial tasks voluntarily for the betterment of the field and their own reputation, editors who are financially dependent on the quality of the journal may be inclined to prioritize acquiring “excellent” articles, leading to a perception that articles from prestigious institutions automatically equate to exceptional research. If the chief editor lacks AI expertise, the environment becomes conducive to the acceptance of repackaged work; most likely smart repackagers do select journals of general reputations but with no special competency in the filed pertinent to the repackaged work.

Yes, repackaged ideas can get published. Whereas conventional plagiarism may still be done and easily get detected [13], smart repackaging is hard to see through.

An Example

Below an example (that I know firsthand) to demonstrate how complicated things can get, and how difficult it may be to spot the repackaging.

• Original paper: Yottixel image search engine [14] introduces a novel Divide & Conquer approach for large histopathology image search. The “Divide” phase involves a multistage clustering technique based on staining/proximity, followed by guided sampling within the clusters. On the other hand, the “Conquer” phase utilizes a DenseNet architecture, followed by feature binarization and matching (see Figure 1).
• Yottixel is published in a journal with established track record in medical image analysis, particularly in image search (1200+ hits when searched for retrieval/CBIR/search), and its editors are experts in the field.
• Repackaged work: Another method, called SISH [15], utilizes the same Divide technique as Yottixel, which is already a major red flag for repackaging. This is why the admission that SISH uses Yottixel’s Divide has been obscured within the paper’s convoluted descriptions. Additionally, SISH employs the same DenseNet and binarization as Yottixel. Thus far, SISH is essentially equivalent to Yottixel. Of course, the repackagers must introduce some modifications to justify claiming ownership of the repackaged model. Consequently, SISH incorporates an autoencoder and a tree for discrete features in parallel with DenseNet and binary values. SISH represents a slightly expanded version of Yottixel (see Figure 1) but is repackaged as a new method.
• SISH is published in a journal with no established reputation in medical image analysis (less than 200 hits when searched for retrieval/CBIR/search; zero for CBIR), and its editor has no track record in AI or in medical image analysis.

One may examine the authors’ affiliations of original and repackaged works, in conjunction with the competencies of the journal editors, to gain more insight. However, as repackaging of a complex AI paper often has intricacies, the interested reader may have to read both papers in detail and any comments (e.g., comments by Yottixel’s authors [16]) before making any conclusive evaluation.

Is repackaging plagiarism?

Does a repackaged paper constitute plagiarism? Technically, no.

Based on conventional definitions repackaging is not plagiarism because the repackagers cite their sources and do not copy/paste text from the original work. However, successful repackagers may, intended or not, steal credits and merits from the original authors and inventors by redirecting attention from the original work to the repackaged paper, which is largely not their own. In this sense, repackaging constitutes a new type of plagiarism that aims at pirating credit and reputation by reselling existing ideas under a different name but citing the plagiarized works to get away with the theft.

The Main Damage of Repackaging

Of course the authors of any original work may see themselves as victims of an insidious theft, and this is most likely true in most repackaging cases. But this is not the main damage that the “smart” repackagers inflict. The most pressing question is not even why major journals with high reputation would publish such works. The actual question is rather why we, as scientists, are driven to steal each other’s credit.

In the era of fast-paced AI competition, we have become ‘publication machines’ obsessed with sensational news, where every single paper must be presented as a breakthrough to impress our employers for sake of promotions and recognitions. Many papers are rejected at major AI conferences for lacking novelty. Our institutions expect us to be at the forefront of developments, acting as avant-guards. All of this pressure fuels our egos and ambitions; ethics becomes of little account .

Can an ambitious scientist be an ethical scientist? Can ethics coexist with ambition and competition? Considering the enormous potential of AI, where will such acquisitive science lead us as a society?

Author: H.R.Tizhoosh

Author’s Disclaimer: This article expresses my assessments as a scientist, and not the views of my present or former employers or any other organization.

References

[1] Grossberg, Michael. History and the disciplining of plagiarism. Originality, imitation, and plagiarism: Teaching writing in the digital age (2008): 159–172.

[2] Pappas, Theodore. “Plagiarism, Culture, and the Future of the Academy.” Humanitas 6, no. 2 (1993): 66–80.

[3] Oxford University Website, Plagiarism,https://tinyurl.com/ynd7z7m4 , visited on June 12, 2023.

[4] Tizhoosh, H.R., The Surge of Sensationalist COVID-19 AI Research, News Medical, May 13, 2020, https://tinyurl.com/2p923jnm

[5] Tizhoosh, H.R., and Jennifer Fratesi. COVID-19, AI enthusiasts, and toy datasets: radiology without radiologists. European Radiology 31, no. 5 (2021): 3553–3554.

[6] Foltýnek, Tomáš, Norman Meuschke, and Bela Gipp. Academic plagiarism detection: a systematic literature review. ACM Computing Surveys (CSUR) 52, no. 6 (2019): 1–42.

[7] Laband, David N., and Michael J. Piette. Favoritism versus search for good papers: Empirical evidence regarding the behavior of journal editors, Journal of Political Economy 102.1 (1994): 194–203.

[8] Colleen Flaherty, When Journals Play Favorites, Inside Higher Ed, 2018, URL: https://tinyurl.com/2ktv82af

[9] Reingewertz, Yaniv, and Carmela Lutmar. Academic in-group bias: An empirical examination of the link between author and journal affiliation, Journal of Informetrics 12.1 (2018): 74–86.

[10] Ipsitaa Khullar, Combatting the privilege of attending elite institutions, London School of Economics and Political Science, 2022, URL: https://bit.ly/3MihJ18

[11] The Economist, Why do American universities favour the children of alumni? 2021, URL: https://tinyurl.com/59w6a6yj

[12] Colleen Flaherty, Publishing’s Prestige Bias, Inside Higher Ed, 2018, URL: https://tinyurl.com/bdepj2wt

[13] Retraction Watch, Controversial AI expert admits to plagiarism, blames hectic schedule, URL: https://tinyurl.com/59s9enw3, visited on June 13, 2023

[14 ] Kalra, Shivam, Hamid R. Tizhoosh, Charles Choi, Sultaan Shah, Phedias Diamandis, Clinton JV Campbell, and Liron Pantanowitz. “Yottixel–an image search engine for large archives of histopathology whole slide images.” Medical Image Analysis 65 (2020): 101757.

[15] Chen, Chengkuan, Ming Y. Lu, Drew FK Williamson, Tiffany Y. Chen, Andrew J. Schaumberg, and Faisal Mahmood. “Fast and scalable search of whole-slide images via self-supervised deep learning.” Nature Biomedical Engineering 6, no. 12 (2022): 1420–1434.

[16] Sikaroudi et al., Comments on “Fast and scalable search of whole-slide images via self-supervised deep learning”, arXiv:2304.08297, https://doi.org/10.48550/arXiv.2304.08297

Comments on “Fast and scalable search of whole-slide images via self-supervised deep learning”

Posted: 2023/07/28 by Admin

Milad Sikaroudi, Mehdi Afshari, Abubakr Shafique, Shivam Kalra, H.R.Tizhoosh

Kimia Lab, University of Waterloo, ON, Canada 2 Rhazes Lab, Mayo Clinic, Rochester, MN, USA

Chen et al. [Chen2022] recently published the article “Fast and scalable search of whole-slide images via self-supervised deep learning” in Nature Biomedical Engineering. The authors call their method “self-supervised image search for histology”, short SISH. The paper is not easily readable, and many important details are buried under ambiguous descriptions.

Incremental modification of Yottixel – Yottixel introduced the concept of “mosaic” through a customized clustering and selection process [Kalra2020a]. While Chen et al. frequently mention “Yottixel” and “mosaic,” they only acknowledge once that they have followed the Yottixel’s mosaic generation process. This inadequate acknowledgment fails to give proper credit to Yottixel’s contributions to their scheme. It is evident that SISH cannot function without the Yottixel mosaic. Unfortunately, Chen et al. do not sufficiently emphasize the reliance of SISH on the Yottixel mosaic. The task of searching in archives of gigapixel WSIs, like any other big-data problem, requires a well-established computer science strategy: “Divide and Conquer.” The Yottixel mosaic serves as the essential “Divide” stage, breaking down the challenging problem of WSI processing into manageable parts, represented by a mosaic of patches. This concept has been extensively validated [Kalra2020b]. SISH has borrowed the crucial Divide element from Yottixel, albeit citing the Yottixel paper, without adequately explaining the significance of the mosaic.

It is important to recognize that WSI search is only possible through the Divide approach, and without a new patching algorithm, no new solution can be achieved.

Not referencing MinMax binarization – Chen et al. state, “The binarization process converts a continuous vector to a binary string by starting from ∞ and then traversing through all elements in the vector to compare the value of the current element with the next one. If the next value is smaller than the current, it assigns the current value to 0, and 1 otherwise.” This description corresponds to the MinMax algorithm [Tizhoosh2016], which was utilized in Yottixel [Kalra2020a] as a simple yet effective approach for computing a 1D approximation of feature gradients. However, Chen et al. neither reference the MinMax paper nor its initial use on deep features [Kumar2018]. Consequently, readers may be under the impression that this method is being introduced by Chen et al. It is worth noting that the MinMax method is a patented technology that has been commercially implemented [Tizhoosh2020].

It is highly irresponsible and misleading to claim that your method is “open source” when crucial components of it (that are not your own) are patented and commercialized.

SISH is a misnomer – Chen et al. employ the term “self-supervised image search.” However, if the usage of self-supervision in training a network does not introduce a new pretext task, it may not qualify as a novel approach. In particular, SISH lacks a new loss function and seems to rely on conventional augmentation methods for training. Additionally, it does not propose any search-oriented loss function. It is important to clarify that the term “self-supervised image search” does not mean that the search algorithm supervises the search queries. Rather, it claims that the SISH paper employs embeddings acquired through self-supervised training. However, strictly speaking, autoencoders do not perform self-supervision. Consequently, the term “self-supervised image search” is misleading and a misnomer. It is highly unlikely that the authors were unaware of the appropriate AI terminology.

In light of these clarifications, it appears that the term ‘self-supervised’ has been included in the title to enhance the perceived novelty for the general readership in the pathology community.

Embeddings – Yottixel uses DenseNet features (pre-trained on natural images), it also mentions that other networks might be used. Combining DenseNet with an autoencoder does not create a new search framework worthy of a new name. The main question remains why the original framework of Yottixel for using a pre-trained DenseNet has not been replaced with the trained autoenoder? Keeping DenseNet (and subsequent barcoding, and before that the Yottixel’s mosaic) clearly shows the attempt to slight modify the original Yottixel chain without any major innovation.

Questions and concerns about experiments/comparisons

• Ranking search results – Modifying search results, such as through additional ranking and classification, constitutes a “post-processing” procedure regardless of the search engine employed. When comparing different search engines, it is imperative to employ the same pre- and postprocessing methods for all search procedures. This holds particular significance since Chen et al. emphasize that “the ranking algorithm plays a crucial role in the success of SISH ”. The classification and sorting of search results is not a novel concept [Ebrahimian2020] and should have been applied to Yottixel’s results as well. Consequently, the reported results, which claim outlandish improvements of 45%, are neither fair nor likely to be reliable.

• Speed – using vEB trees [Boas1975] and other logarithmic data structures are quite trivial and part of any efficient implementation [Friedman1977]. Talking about theoretical upper bounds, O(n) versus O(log n), seems to be in negligence of the reality of high-grade commercial implementation of any such engine. Claiming “theoretical constant time”, in light of massive memory usage by SISH is another attempt to increase the perceived novelty. Besides, any vector-based search can be implemented efficiently. For instance, in one implementation of IrisCodes [Daugman2015] “800 trillion (8 × 1014) cross-comparisons are performed every day” by smart memory usage of “great speed of Exclusive-OR (XOR) IrisCode matching, which executes at millions/sec per CPU single core”, at a time where GPUs and multi-core parallel processing were not available.

Borrowing existing concepts and labeling the modified version with a new name is simply repackaging. Searching in medical archives is a formidable task that necessitates years of research. Ideas and innovations from all active researchers are crucial. Regrettably, Chen et al.’s work is essentially a repackaging of Yottixel under a different name. While it is acceptable to draw inspiration from each other’s work, it is crucial to acknowledge that repackaging unfairly redirects credits and recognition to the authors of the repackaged work. Upholding academic integrity and honesty entails recognizing that only the original authors of a work have the authority to rename its incremental modifications.

References

[Chen2022] Chen, Chengkuan, Ming Y. Lu, Drew FK Williamson, Tiffany Y. Chen, Andrew J. Schaumberg, and Faisal Mahmood. ”Fast and scalable search of whole-slide images via self-supervised deep learning.” Nature Biomedical Engineering 6, no. 12 (2022): 1420-1434.

[Kalra2020a] Kalra, S. et al. (2020). Yottixel–an image search engine for large archives of histopathology whole slide images. Medical Image Analysis, 65, 101757.

[Kalra2020b] Kalra, S. et al., 2020. Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence. NPJ digital medicine, 3(1), p.31.

[Boas1975] Peter van Emde Boas: Preserving order in a forest in less than logarithmic time (Proceedings of the 16th Annual Symposium on Foundations of Computer Science 10: 75-84, 1975)

[Friedman1977] Friedman, J. H., Bentley, J. L., & Finkel, R. A. (1977). An algorithm for finding best matches in logarithmic expected time. ACM Trans. on Mathematical Software, 3(3), 209-226.

[Ebrahimian2020] Ebrahimian, A. et al. (2020). Class-aware image search for interpretable cancer identification. IEEE Access, 8, 197352-197362.

[Daugman2015] Daugman, J. Information theory and the iriscode. IEEE TIFS, 11(2), 400-409, 2015.

[Tizhoosh2016] Tizhoosh, Hamid R. et al. “Minmax radon barcodes for medical image retrieval.” International Symposium on Visual Computing, pp. 617-627. Springer, Cham, 2016.

[Kumar2018] Kumar, M. D., Babaie, M., & Tizhoosh, H. R. Deep barcodes for fast retrieval of histopathology scans. Int. Joint Conf. on Neural Networks (IJCNN), pp. 1-8, 2018.

[Tizhoosh2020] Tizhoosh, H. R. (2020). U.S. Patent No. 10,628,736. Washington, DC: U.S. Patent and Trademark Office

“Source: https://doi.org/10.48550/arXiv.2304.08297″ 

Kimia Lab move to new location complete!

Posted: 2022/01/04 by Admin

Kimia Lab, founded in 2013, has been housed in many different buildings; DWE, EC2, and lately in Engineering 7. The dispersed work in several rooms in Engineering 7 was not ideal for Kimia Lab’s intensive research needs. The lab has been in the “Engineering 7” building since 2017. Therefore, the decision was made to move to a new place where Kimia lab’s researchers could work in one single area. The moving request was approved by the Faculty of Engineering, assigning Kimia Lab a large classroom in one of the oldest buildings on campus, namely “Engineering 2”.

It took almost over two years for our new lab space to be renovated and brought to new standards during the pandemic era. All the renovations have been completed by the end of November 2021. A recently published video provides a glimpse into Kimia Lab and its researchers in the new location.

Watch a video from the new Kimia space

AI @ Mayo Clinic

Posted: 2021/12/02 by Admin

Professor Tizhoosh leaves for Mayo Clinic

Professor Tizhoosh, the Director of Kimia Lab, is moving to the United States soon to join Mayo Clinic. He is a new member of the Department of Artificial Intelligence and Informatics as a Professor of Biomedical Informatics.

“The needs of the patient come first is the guiding maxim everywhere on Mayo campuses”. says Dr. Tizhoosh, “Serving patients as a computer scientist by assisting world-class Mayo physicians is undoubtedly a dream coming through for me.”

Mayo Clinic’s Department of Artificial Intelligence and Informatics includes clinical and research faculty, as well as operational staff who engage in the use of informatics and digital technology to improve human health. The department promotes collaborative research for advancing the methods, applications and infrastructure in digital medicine with the emphasis on the synergy of people, processes and technology, representing a multidisciplinary unification based on biomedical, computational and social sciences.

“Kimia Lab will be a major part of my research at Mayo Clinic,” adds Dr. Tizhoosh, “we will build the most advanced search engine to assist Mayo Clinic’s physicians in research, diagnosis and treatment planning.”

Pathology News – Conversation with Dr. Hamid Tizhoosh, Founder of KIMIA Lab and Leading Expert in the Development of Unsupervised AI for Tissue Pathology

Posted: 2021/09/07 by Admin

Pathology News, Jonathon Tunstall, interviewed Dr. Hamid Tizhoosh on June 21, 2021. The interview has been published on the Pathology News website on Sept 6, 2021.

“What we have to learn from day one when we design these AI applications, is that pathology has to come with us. We cannot just design a network as computer scientists and then go to the pathologists just when we need to validate it. The pathologist has to be with us from the start.” 

“…as human beings, we have a shared consciousness. Our needs and deepest desires are the same and the psychological profile of homo sapiens is the same anywhere you go. This cloud system that you mention would be a new manifestation of human consciousness. Yes, this will happen. No one can prevent that, because it is what happens when you have shared wisdom and knowledge. When humans invented fire, it was quickly shared around the planet and in the same way, we publish papers today and we share our knowledge and influence each other. With the cloud as a new concept in shared consciousness and knowledge, no one knows in which direction that will take us, but we will go that way, and nobody can prevent it. It will definitely bring about its own requirements and conditions and it will open up new problems so we will see things that we have not seen before. Diagnosing carcinomas will become almost trivial, it will be like suddenly being able to access the 4th dimension in our three-dimensional perception. Can we get to this point just by talking about the data? Maybe we can, but we have to come up with a more efficient way of sharing our knowledge.”

The full interview can be found on Pathology News.

Waterloo Researchers are Accelerating the Cancer Diagnosis Process via Artificial Intelligence (AI)

Posted: 2021/07/26 by Admin

For many pathologists, diagnosing cancer is one stressful and complicated process.

Typically, these medical professionals are alone in their office, examining a biopsy sample through a microscope. Only in major research hospitals may they have the privilege of consulting with other colleagues for a second opinion. In only very doubtful cases, they may request a teleconsultation. Although printed or digital atlases containing thousands of sample cancer images approved by professionals such as the World Health Organization may be helpful, they may not be up-to-date making the diagnosis process time consuming and inefficient.

Unfortunately, not every medical professional has this resource, especially in remote locations across the world such as Latin America and Africa. Pathologists are unable to make proper assessments in these countries with poor health conditions and greater need for high quality medical resources.

Now, Hamid Tizhoosh, Professor of Systems Design Engineering at the University of Waterloo wants to change that. As Director of the Laboratory for Knowledge Inference in Medical Imaging Analysis (Kimia Lab), he is leading a joint project with the International Collaboration for Cancer Classification and Research (IC3R). This new consortium is co-ordinating research on tumour classification and cancer diagnosis. An international collaboration with partners in France, Belgium, Italy, the United States, Australia, Japan, and Singapore, the Kimia Lab is the only Canadian partner. IC3R’s main mission is to provide a faster and more accurate way to diagnose and treat numerous diseases and to establish new quality assurance through information extraction.

Recently, IC3R have started to collaborate on creating a search engine that contains WHO archives of breast cancer samples. Professor Tizhoosh says it contains 39 different types of common and rare abnormalities, some that are cancer and some that are benign. The user scans the biopsy sample, and the search engine will identify the closest sample picture, quantifying the tissue similarity. Within seconds, pathologists can find the most likely diagnosis, thus accelerating the diagnosis process, especially for those in remote places.

How did this collaboration evolve? While attending the annual meeting of the United States and Canadian Academy of Pathology (USCAP), the largest pathology conference in America, Professor Tizhoosh demonstrated a search engine to fellow researchers from America. Through discussion, they realized they had a lot of “synergy that they could explore together.”

Soon after, IC3R began to collect images from Grand River Hospital for external validation while simultaneously receiving support from renowned universities such as the University of Michigan and the University of British Columbia. After launching the invention, Kimia Lab, which combines medical archives and machine learning for medical diagnosis and analysis, started to test and use it. The pilot project is currently focused on breast cancer diagnosis, but researchers are open to exploring diverse types of cancers in the near future.

Modified UW Story. Original text: By Mayuri Punithan – (Office of Research) at University of Waterloo

Kimia Lab Members Selected as Vector Postgraduate Affiliates

Posted: 2021/03/31 by Admin

The Vector Institute announced the 2021 candidates for its Postgraduate Affiliate Program. The new cohort of 30 talented researchers is a combination of new and returning members to Vector’s growing community. Established in 2018, the program promotes engagement and collaboration among researchers in the AI community who are in the early stages of their careers. Morteza Babaie and Shivam Kalra from Kimia Lab are among the new affiliates.

Made up of graduate students and postdoctoral fellows, members of the 2021 cohort represent universities and institutions across Ontario. Their research areas span core machine learning, neuroscience, health, computational linguistics, natural language processing, computational biology, computer vision, fairness, photonics for AI, systems, and how people relate to and understand AI. 

They join the 16 Postgraduate Affiliates selected in 2020 who are completing the two-year term of their appointments as well as Vector’s vibrant and talented community of over 500 researchers.

Postgraduate Affiliates are selected through a competitive process; applicants are evaluated and selected based on the strength of their past research contributions and the alignment of their interests with Vector’s vision, mission, and research.

Shivam Kalra is a Ph.D. candidate at the Kimia Lab. He has completed his MSc at the University of Waterloo and BSc in Software Engineering at the University of Ontario Inst. of Technology. Shivam is an outstanding student and a researcher, and the holder of various national, provincial, and university-level scholarships and awards. He was selected as the finalist for the Governor’s General Gold Medal from the University of Waterloo for his excellent academic standings and research contributions. During the Ph.D., Shivam has developed a search engine for digital pathology archives called Yottixel.  The search technology for pathology archives, such as Yottixel is a pioneering step for tapping the immense potential of AI that can revolutionize biomedical research for infectious diseases and cancer. For the rest of his Ph.D., Shivam is interested to research in federated learning for computational pathology. Federated learning is a privacy-preserving machine learning paradigm that enables multi-institutional collaborations on collaborative diagnostic and treatment projects without disclosing patient data. Putting privacy at the forefront of AI has the potential to transform the way AI research is conducted in biomedical fields. 

Morteza Babaie is a postdoctoral fellow and the lab manager at the Kimia Lab since 2018. He joined Kimia Lab as a Ph.D. visiting scholar in 2016 and started working on medical image analysis. Since then, he has been the author or co-author of 30 research papers in computational medicine journals and conferences with near 500 google citations and an H-index of 12. His main research interests include image processing, machine learning, and AI.

“I am incredibly proud to work at Kimia Lab, especially when I noticed 2 out of 28 Vector-affiliated postgraduates in 2021 were selected from our Lab, which shows the direction, impact, and quality of  Professor Tizhoosh’s leadership”, says Dr. Babaie.

Modified Vector news. Original text: Ian Gormely

Vector Welcomes New Researchers to Po

Kimia Students receive Waterloo AI Institute Scholarships

Posted: 2021/02/25 by Admin

The Waterloo AI Institute provides twelve one-time awards valued at $5,000 each. One Master’s student and one Ph.D. student from each of the University’s six faculties were chosen to receive a scholarship. This year Shivam Kalra and Milad Sikaroudi, Ph.D. students from Kimia Lab have been awarded the AI scholarship.

Shivam Kalra is a Ph.D. candidate at the Kimia Lab. He has completed his MSc at the University of Waterloo and BSc in Software Engineering at the University of Ontario Inst. of Technology. Shivam is an outstanding student and a researcher, and the holder of various national, provincial, and university-level scholarships and awards. He was selected as the finalist for the Governor’s General Gold Medal from the University of Waterloo for his excellent academic standings and research contributions. During the Ph.D., Shivam has developed a search engine for digital pathology archives called Yottixel.  The search technology for pathology archives, such as Yottixel is a pioneering step for tapping the immense potential of AI that can revolutionize biomedical research for infectious diseases and cancer. For the rest of his Ph.D., Shivam is interested to research in federated learning for computational pathology. Federated learning is a privacy-preserving machine learning paradigm that enables multi-institutional collaborations on collaborative diagnostic and treatment projects without disclosing patient data. Putting privacy at the forefront of AI has the potential to transform the way AI research is conducted in biomedical fields. 

Milad Sikaroudi is currently pursuing a Ph.D. degree with the Department of Systems Design Engineering and is a graduate research assistant at the KIMIA Lab, University of Waterloo. He has published several impactful publications in using AI for medical image analysis.

“I am honored to be part of the Waterloo AI institute community taking steps in the right direction of translational requirements for deploying AI”, says Milad.

A search engine for better disease diagnosis and treatment

Posted: 2021/02/09 by Admin

Waterloo Engineering researcher partners with alumnus on new technology they hope will revolutionize health care

Hamid Tizhoosh was looking for a new idea, a fresh start when he began talking to doctors about how they do their jobs and how they might do them better. 

Six months into his consultations, with his engineering lab at the University of Waterloo reduced to a one-man show by a failed artificial intelligence (AI) startup, he heard something that almost floored him. 

Pathologists in the 21st century still rely on atlases — books of images from biopsy samples — and flip through them for potential matches to help diagnose new cases. Really? Books of old images? That was it, the spark that sent the Systems Design engineering professor roaring down a productive new research path. “They were using a very Stone Age type of search,” he recalls. “When I learned that, I said, ‘For heaven’s sake, we should do this automatically. It is an image search. Computers can do it.’”  

Seven years later, Tizhoosh has turned that basic concept into new technology he hopes will revolutionize health care by giving doctors a simple, powerful tool to help diagnose, treat and research disease via search in large medical image archives. 

Partnering with industry to secure $3.14 million 

And working with him to realize that goal is a local company, Huron Digital Pathology of St. Jacobs, that he approached for backing when his early work started showing promise. 

Huron is the industrial partner in a consortium led by Tizhoosh and researchers in his Laboratory for Knowledge Inference in Medical Image Analysis (KIMIA Lab) which secured $3.14 million in funding through the Ontario Research Fund: Research Excellence program in 2018. 

The money is important, but both sides stress the relationship goes well beyond Huron providing $500,000 (cash and inkind) for research over five years in exchange for commercialization rights. 

In addition to being a shareholder and AI advisor to the company, Tizhoosh helped hire its engineers and rolled up his sleeves to work booths at conferences and trade shows. 

“We lift the KIMIA lab up, the KIMIA lab lifts us up and, of course, the University of Waterloo is a fantastic calling card,” says Patrick Myles (BA ’87), the CEO of Huron. “We open doors for each other. 

“And on a personal level, Hamid and I almost finish each other’s sentences. We’re both out there telling the story of how this wonderful technology can make lives better and doctors more efficient.” 

Search engine combs archives for close matches 

The technology at the heart of the partnership is essentially a specialized search engine that allows doctors to comb archives of digital images of tissue samples for the closest matches to new cases. 

The original images contain massive amounts of digital data, so the search is only possible because Tizhoosh came up with a way of using AI to identify key features and convert them into bunches of barcodes. 

That reduces the size of images to a tiny fraction of the originals and indexes them, enabling the search engine to find matches in archives of millions of images in a split second using ordinary computers. 

“We designed the search from the beginning to be super-efficient, to do this without heavy-duty computational power,” Tizhoosh says. 

By finding similar images, the search engine instantly connects doctors to a treasure trove of information on old cases — the diagnosis report, the treatment plan, the eventual outcome — that is now just sitting in archives. 

Myles compares it to turning a giant pile of random books into a library structured and organized using the old Dewey Decimal System. 

“Our search engine unlocks all of the data that already exists,” he says. “It’s really a knowledge-sharing tool.” 

A hint of its potential came last fall when the KIMIA Lab was selected by the World Health Organization to contribute to a global research project on cancer categorization using its image-retrieval technology. 

Waterloo tech will modernize the world’s largest tissue archive 

Around the same time, the United States military signed on as Huron’s first paying customer to modernize the Joint Pathology Centre, home to the largest collection of preserved tissue samples in the world. 

That deal put the project two years ahead of schedule in terms of commercialization, but Tizhoosh is convinced they have still only scratched the surface. 

He looks forward to the day the system is used everywhere — including areas of the developing world where pathologists are especially scarce — to virtually eliminate diagnostic error, personalize treatment plans, and fuel drug development. 

“I think we’re only at the very beginning, to be honest,” Tizhoosh says. “I believe this is a disruptive technology that will eventually touch every area of the medical field. 

“Even we don’t know what the full impact of image search in medical archives will be. Every time we talk to doctors, they give us new ideas.” 

Modified UW Story. Original text: By Brian Caldwell – Faculty of Engineering

Kimia Labs Research Featured in UWs Global Impact Report:

Vector Welcomes 2020 Cohort of Faculty Affiliates

Posted: 2021/01/06 by Admin

Professor Hamid Tizhoosh, the director of Kimia Lab, has been appointed as a faculty affiliate to the Vector Institute for a second time. On December 22, 2020, The Vector Institute announced that 72 faculty holding appointments at universities across Ontario have been appointed as 2020 Faculty Affiliates. The 2020 cohort is a combination of new and returning members. Dr. Tizhoosh’s first appointment was in June 2018 (the first Vector cohort of faculty affiliates) and ended in December 2020. The new appointment will end in December 2022.

The Vector Institute, based in Toronto, is one of the world’s top research institutes in machine learning (ML) and deep learning (DL). The daily life and concentration of expertise at Vector fosters a network of over 500 researchers and potential collaborators. Researchers at Vector work with industry sponsors as well as universities and other public institutions to support the artificial intelligence (AI) ecosystem in Ontario. The Vector Faculty Affiliates Program is intended to expand the research community’s expertise in the areas of AI, computer science, engineering, and other disciplines related to machine learning, as well as strategic domains of application. Vector Faculty Affiliates are appointed to a two‐year term and have part-time access to resources. Faculty Affiliates form an integral part of the Vector community, engaged in both research and community activities. Where interests align, Faculty Affiliates can also benefit from interacting with industry sponsors, as well as health and academic partners, through participation in networking events, seminars, training sessions and workshops.

Kimia Lab joins WHO Consortium for Cancer Research

Posted: 2020/11/04 by Admin

November 4, 2020 – UW’s Kimia Lab will contribute to WHO’s global research for cancer categorization.  The International Collaboration for Cancer Classification and Research (IC3R) officially accepted Kimia Lab as a full member on October 22, 2020. The UW lab, which is specialized on image search in medical archives, will assist IC3R to implement content-based image retrieval for a  WHO global digital atlas of histopathology images. The project will start with a pilot project for breast cancer in 2021.

The International Agency for Research on Cancer (IARC), together with several international partners, has established IC³R to promote evidence-based practice and standards for cancer classification and research. IC³R will help to address challenges related to tumour classification, such as its increasingly multidimensional nature, the overwhelming amount of scientific information available in this area, and the translation of research findings into tumour classification and cancer diagnoses. IC3R is an initiative of the WHO Classification of Tumours Group (WCT) at IARC, which produces the WHO Classification of Tumours series used by pathologists and cancer researchers worldwide. WCT is uniquely situated to initiate an international collaboration to assess tumour classification and its relevant issues. IC3R will provide a forum to coordinate evidence generation, synthesis, evaluation, and standard setting for tumour classification.

Some other members of the IC³R Steering Group are IARC; the Molecular Diagnostics Pathology Department at Centre Léon Bérard, in Lyon, France; the Istituto Nazionale Tumori “Fondazione Pascale” in Naples, Italy; the Cancer Centre of Sciensano, in Brussels, Belgium, and National Cancer Center, Japan. Some of the associated entities also include the American Society of Clinical Oncology (ASCO), the International Collaboration on Cancer Reporting (ICCR), and the National Cancer Registry and Analysis Service, Public Health, England.  

The Laboratory for Knowledge Inference in Medical Image Analysis, short Kimia Lab, established in Fall 2013 at the University of Waterloo, conducts research at the forefront of mass image data in medical archives using machine learning schemes with the ultimate goal of extracting information that cannot only support a more speedy and accurate diagnosis and treatment of many diseases but also, and more significantly, establish new quality assurance based on mining of collective knowledge of medical experts. Kimia Lab is presently home to 14 Ph.D. students, 4 MSc students, 3 research assistants, and a postdoctoral fellow. The lab’s director, Professor Hamid Tizhoosh is an expert in medical image analysis and artificial intelligence with more than 25 years of academic and industrial experience. Kimia Lab collaborates with several pathology departments at multiple research institutions, among Ohio State University (Dr. Anil Parwani), University of Michigan (Dr. Liron Pantanowitz), University Health Network (Dr. Phedias Diamandis),  Hamilton Health Sciences (Dr. Clinton Campbell), and Grand River Hospital (Dr. Adrian Batten).

World’s largest human tissue archive adopts Kimia’s AI algorithms

Posted: 2020/10/26 by Admin

Artificial intelligence technology will be used to help modernize US federal pathology facility

Waterloo, ON, October 26, 2020 – Technology developed by Kimia researchers has been adopted by a major pathology facility in the United States.

The Joint Pathology Center (JPC), which has the world’s largest collection of preserved human tissue samples, will use an artificial intelligence (AI) search engine to index and search its digital archive as part of a modernization effort.

Development of the system was led by researchers at the Laboratory for Knowledge Inference in Medical Image Analysis (KIMIA Lab) at Waterloo. It was commercialized by partner Huron Digital Pathology of St. Jacobs.

The image retrieval technology, with the scientific name Yottixel, allows pathologists, researchers and educators to search large archives of digital images to tap into rich diagnostic data.

To be used for biomedical research

It will be used to enhance biomedical research for infectious diseases and cancer, enabling easier data sharing to facilitate collaboration and medical advances.

The JPC is the leading pathology reference centre for the US federal government and part of the US Defense Health Agency

In the last century, it has collected more than 55 million glass slides and 35 million tissue block samples. Its data spans every major epidemic and pandemic, and was used to sequence the Spanish flu virus of 1918. It is expected that the modernization also helps to better understand and fight the COVID-19 pandemic.

A Globally Unique Repository 

“We are delighted to see that our algorithms are about to explore the world largest digital archive of biopsy samples” Professor Hamid Tizhoosh, the Director of Kimia Lab, says, “we will continue to design and commercialize novel AI solutions for the medical field. The opportunity comes with unprecedent challenges that need fresh ideas and established competency to fully exploit the big data for the diagnostic imaging of future.”

Researchers at Waterloo have obtained promising diagnostic results using their AI search technology to match digital images of tissue samples in suspected cancer cases with known cases in a data base.

In a paper published earlier this year, a validation project led by Kimia Lab achieved accurate diagnoses for 32 kinds of cancer in 25 organs and body parts.

“We showed it is possible using this approach to get incredibly encouraging results if you have access to a large archive,” said Hamid Tizhoosh, a systems design engineering professor and director of the KIMIA Lab. “Image search is undoubtedly a platform to intelligently explore big data and enable what we call virtual peer review: It is like putting many experts in a virtual room and having them reach consensus.”

Kimia Lab and Huron at Pathology Visions 2020

Posted: 2020/10/01 by Admin

Monday, October 26 | 9-9:45am PT
Live Q&A: 9:45-10:05am PT

In this pre-conference workshop, Huron Digital Pathology’s CEO, Patrick Myles, and their AI advisor, Professor Hamid Tizhoosh, will illustrate how Huron’s Lagotto^TM image search platform has been designed to integrate seamlessly into existing and emerging digital pathology workflows, describing several workflow scenarios.

Attendees will learn the following:

• How does WSI indexing work? What file formats does Lagotto support (does it work on scanners other than Huron’s)?
• How long does it take to index a large WSI archive with millions of slides? What about new slides as they enter the archive?
• How does Lagotto’s API integrate with 3^rd party image management systems? What about 3^rd party AI algorithms from various vendors? Laboratory information systems?
• What does an on-premise deployment of image search look like? What about cloud deployment?

Patrick Myles
CEO
Huron Digital Pathology
Patrick Myles is the CEO of Huron Digital Pathology, a Canadian medical device and software company pioneering image search to connect pathologists to the vast knowledge contained in the world’s pathology reports. Prior to joining Huron, Patrick spent 18 years at Teledyne DALSA, most recently as Vice President of Business Development. He is board member of the Digital Pathology Association.

Hamid Tizhoosh, PhD
Kimia Lab, Director
University of Waterloo
Dr. Hamid R. Tizhoosh is a Professor in the Faculty of Engineering at the University of Waterloo since 2001 where he leads the KIMIA Lab (Laboratory for Knowledge Inference in Medical Image Analysis). He is the author of two books, 14 book chapters, more than 150 journal and conference papers, and multiple patents. He is the AI Advisor of Huron Digital Pathology and is a faculty affiliate to the Vector Institute in Toronto.

Register for PV20 Virtual Today!

:: https://digitalpathologyassociation.org/huron-preconference-workshop

KimiaNet at Pathology Visions 2020

Professor Tizhoosh will also talk at the conference in a separate session to report recent progress in Kimia Lab on training deep networks for digital pathology. The talk is titled “KimiaNet – Training a Histopathology Deep Network from Scratch“. The main objective of the talk is to show how unlabelled data like TCGA images can be used to train a deep network.

Deep embeddings, or feature vectors, provided by pre-trained deep artificial neural networks have become a dominant source for image representation in digital pathology. Their contribution to the performance of image analysis can be improved through fine-tuning. One might even train a deep network from scratch with the histopathology images, a highly desirable option which is generally impeded in pathology by lack of labeled images and the computational expense. We propose “KimiaNet” that employs the topology of the DenseNet with four dense blocks, fine-tuned and trained with histopathology images in different configurations. We used more than 240,000 image patches with 1000×1000 pixels acquired at 20x magnification through our proposed “high-cellularity mosaic” approach to enable the usage of weak labels of 7,126 whole slide images of formalin-fixed paraffin-embedded human pathology samples publicly available through The Cancer Genome Atlas (TCGA) repository. We tested KimiaNet using three public datasets, namely TCGA, endometrial cancer images, and colorectal cancer images by evaluating the performance of search and classification when corresponding features of different networks are used for image representation.

:: https://digitalpathologyassociation.org/hamid-tizhoosh

UW Professor on The Pathologist’s “Power List 2020”

Posted: 2020/08/06 by Admin

UW’s Hamid Tizhoosh is among the 20 pathologists and computer scientists listed in this year’s “Power List” of The Pathologist who achieved  Big Breakthroughs, “ trailblazers working at the cutting edge and driving forward the future of the field”.

The magazine collects nominations from the pathology community. An expert panel selects the distinguished physicians and researchers based on their track record and achievements in the field. The Pathologist, an award-winning international monthly publication, is the exclusive print and online media of the American Society for Clinical Pathology (ASCP). The magazine provides ASCP members with access to news, editorial features and opinion pieces on all aspects of laboratory medicine and diagnostics, including the research, personalities and policies that shape the pathology and laboratory medicine sector.

Hamid Tizhoosh is a Professor in the Faculty of Engineering, Systems Design Engineering since 2001 where he leads the KIMIA Lab (Laboratory for Knowledge Inference in Medical Image Analysis). His research activities encompass artificial intelligence, computer vision, and medical imaging. He has developed algorithms for medical image filtering, segmentation, and search. Presently, he is the AI Advisor of Huron Digital Pathology, St. Jacobs, ON, Canada. As well, he is a faculty

Kimia Lab Students publish and present at ECCV and CVPR

Posted: 2020/07/16 by Admin

A novel method developed by graduate students from Kimia Lab, Waterloo Engineering, Mohammed Adnan (1st-year MASc.), and Shivam Kalra (2nd year Ph.D.) has the potential to have a major impact in histopathological image analysis and cancer diagnostics. Their technique uses artificial intelligence (AI) to render digital representations of extremely high-resolution biopsy images. These digital representations can be used for real-time image search, providing critical information to pathologists for well-informed and evidence-based diagnosis. Two of their works have been recently published in top machine learning conferences:

1) Representation Learning of Histopathology Images using Graph Neural Networks, CVPR 2020 (CVMI Workshop)
2) Learning Permutation Invariant Representations Using Memory Networks, ECCV 2020

Diagnostic consensus for cancer is possible through image search using AI

Posted: 2020/03/11 by Admin

Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence

A new system combining artificial intelligence (AI) with human knowledge promises faster and more accurate cancer diagnosis.

The powerful technology, developed by a team led by engineering researchers at the University of Waterloo, uses digital images of tissue samples to match new cases of suspected cancer with previously diagnosed cases in a database.

In tests using the largest publicly available archive in the world – comprised of about 30,000 digitized slides from almost 11,000 patients – the technology achieved up to 100-per-cent accuracy for 32 forms of cancer in 25 organs and body parts.

“AI can help us tap into our medical wisdom, which at the moment is just sitting in archives,” said Hamid Tizhoosh, director of the Laboratory for Knowledge Inference in Medical Image Analysis (KIMIA Lab) at Waterloo. “When you use AI like this, its performance is astounding.”

The system utilizes AI to search digital images of biopsies from confirmed cancer cases for those most similar to a new digital image in an undiagnosed case.

Based on the known, verified findings of the majority of similar images, the system recommends a diagnosis for the new case.

Conducted over four months using high-performance computers and data storage, the tests achieved accurate diagnoses for everything from melanoma to prostate cancer.

“We showed it is possible using this approach to get incredibly encouraging results if you have access to a large archive,” said Tizhoosh. “It is like putting many, many pathologists in a virtual room together and having them reach consensus.”

The archive used in the study, part of a five-year project backed by $3.2 million in funding from the Ontario government, was provided by the National Cancer Institute in the United States.

More work is needed to analyze the findings and refine the system, but Tizhoosh said the results so far demonstrate it has potential as a screening tool to both speed up and improve the accuracy of cancer diagnoses by pathologists.

And in the developing world, it could save lives by enabling remote access to inexpensive diagnosis.

“This technology could be a blessing in places where there simply aren’t enough specialists,” Tizhoosh said. “One could just send an image attached to an email and get a report back.”

Project sponsor Huron Digital Pathology of St. Jacobs, Ontario is currently working to commercialize the technology.

A paper on the research, Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence, appears in the journal Nature Digital Medicine.

Kimia Lab Presents Posters at USCAP 2020

Posted: 2020/02/27 by Admin

The USCAP 109th Annual Meeting

United States and Canadian Academy of Pathology (USCAP) will hold its 109th Annual Meeting for the first time in Convention Center Los Angeles, California, February 29-March 5, 2020. Kimia Lab will present posters on Detecting Specimen Contamination in Whole Slide Imaging Using Artificial Intelligence and Automatic Assessment of Tumor Cellularity in Histopathology Images Using Weakly-Supervised segmentation.

Poster VI – Wednesday March 4 – Informatics
1:00 PM – 4:00 PM
Poster Board Number: 285
LACC West Exhibit Hall A

Poster VI – Wednesday March 4 – Informatics
1:00 PM – 4:00 PM
Poster Board Number: 287
LACC West Exhibit Hall A

Shopify Data Talks – The History of AI

Posted: 2020/02/19 by Admin

Welcome to the first of the Shopify Data Talks series! Join us with Dr. Hamid Tizhoosh as he takes us through the past, present, and future of AI. Tackling questions like when was modern AI born?, why did AI experience a renaissance?, what happens next?, and what should we do now?

We will be gathering at Shopify’s 57 Erb St. location in Waterloo with time for networking, drinks, and hors d’oeuvres before the talk begins.

5:00pm: Registration

5:30pm: Welcome

5:40pm: Drinks and Networking

6:30pm: Speaker

7:00pm: Fireside chat and Q+A

Dr. Tizhoosh is a professor at the University of Waterloo’s Faculty of Engineering where he leads the KIMIA Lab (Laboratory for Knowledge Inference in Medical Image Analysis). His research activities encompass artificial intelligence, computer vision, and medical imaging, and he has developed algorithms for medical image filtering, segmentation, and search. He is the author of two books, 14 book chapters, and more than 140 journal and conference papers. Prof. Tizhoosh also has extensive industrial experience, including more than 10 years of experience in commercialization and start-ups. As CEO of Segasist Technologies, a start-up that developed image segmentation software for radiation oncology, he planned and successfully managed an FDA 510k submission for computer-aided detection of prostate cancer. He is currently the AI Advisor of Huron Digital Pathology, St. Jacobs, ON, Canada.

Our office is fully accessible and we have gender neutral facilities available on site. Should you require any additional accommodations, please let us know as soon as possible.

Artificial intelligence to give Toronto doctors a 2nd opinion

Posted: 2019/11/06 by Admin

Physicians may soon be able to get a second opinion on your diagnosis from a form of homegrown artificial intelligence (AI).

Toronto’s University Health Network (UHN) is teaming up with researchers at the University of Waterloo and the Vector Institute to develop software that can read and provide feedback on medical images like x-rays and ultrasounds.

The groundbreaking technology is specifically aimed at assisting radiologists and pathologists.

“We are trying to basically go after the biggest problems of medical imaging, which we call user variability,” said Dr. Hamid Tizhoosh, director of the KIMIA lab at the University of Waterloo. “Different radiologists may have a different diagnosis looking at the same image – and it seems that AI can assist in removing that variability.”

The AI software will build on an existing program at the UHN called “Coral Review,” which allows physicians to quality check each other’s work. The program was spearheaded by the medical research organization’s senior director, Leon Goonaratne.

“Rather than looking at doing a second pair of eyes on two or three percent of cases, it would be great if we can do it on 100 per cent of cases,” Goonaratne said.

Preliminary work on the software has already been completed. Now, UHN is using about 25,000 high-quality x-ray images, and a specialized group of radiologists, to train the AI’s algorithm.

Goonaratne said the technology is not meant to replace a doctor’s role in diagnosis.

“It’s still a radiologist or pathologist that is taking that input from the algorithm, the software, and ultimately they’re interpreting whether this makes sense or not.”

The initiative is one of a series of Pathfinder projects at Toronto’s Vector Institute for Artificial Intelligence. The aim is to see how AI technology can be effectively integrated into healthcare.

The organization has partnered with multiple groups in the health sector.

Their newest project is using AI to help save a premature infant’s life by identifying when sepsis may occur. Another project is developing a phone application people can use to identify a tick and check if there’s a risk for Lyme Disease.

“When we think about the health sector overall, we just have so much data that’s generated because we’ve moved away from paper processes; and a lot of it isn’t used at all,” said Alison Paprica, Vector’s VP of Health Strategy. “There’s an opportunity to bring computer scientists, AI, and machine learning in to turn that data into actionable knowledge. That’s what the Pathfinder projects each do.”

UHN is hoping to have its AI radiology program roll out within the next 18 months. It will initially be used within the UHN’s network of hospitals.

If it’s successful, it could make its way into healthcare facilities using Coral Review across the province.

:: Source: https://toronto.citynews.ca/2019/11/04/artificial-intelligence-second-opinion/

AI system more accurately identifies collapsed lungs using chest x-rays

Posted: 2019/10/16 by Admin

WEDNESDAY, OCTOBER 16, 2019 Toronto, Ontario, Vector Institute’s Evolution of Deep Learning Symposium.

New assistive technology can diagnose collapsed lungs from chest x-rays with a higher degree of accuracy than radiologists.

The system, developed at the University of Waterloo, uses artificial intelligence (AI) software to search a huge database of x-ray images with known diagnoses for comparison to x-rays of new patients with unknown conditions.

That approach enables researchers to identify 75 per cent of cases of collapsed lungs, or pneumothorax. On average, medical specialists diagnose fewer than 50 per cent of cases when chest x-rays are used.

“Our results are very exciting,” said Antonio Sze-To, a postdoctoral fellow at Waterloo. “The AI we use works almost like magic – and it will help radiologists save lives.”

Pneumothorax occurs when air gets between the chest wall and the outside of a lung, causing pain and symptoms such as shortness of breath. If left undiagnosed and untreated, it can suddenly deteriorate and cause death.

Hamid Tizhoosh, a professor of systems design engineering and director of the Laboratory for Knowledge Inference in Medical Image Analysis (KIMIA Lab), said the most serious cases of collapsed lung are relatively easy for experienced radiologists to diagnose from x-rays.

Minor cases are extremely challenging to spot, however, resulting in missed diagnoses that put up to 50 per cent of patients at risk.

“We spend a lot of time, energy and resources needlessly investigating other possible causes of the same symptoms and people suffer in the meantime,” Tizhoosh said.

Researchers are working with the University Health Network (UHN), a healthcare and medical research organization consisting of several Toronto-area hospitals, on a project backed by the Vector Institute, a not-for-profit corporation dedicated to advancing AI.

The goal is to increase the accuracy of the technology to over 90 per cent and integrate it next year into a software system, Coral Review, developed and used at UHN-affiliated hospitals. If successful at UHN, it would be offered to other hospitals now using the system.

Coral, a quality assurance tool, allows doctors to provide second opinions by reviewing medical imaging diagnoses made by their peers.

“We’re very excited about the collaboration with the University of Waterloo and the Vector Institute,” said Leon Goonaratne, Senior Director at UHN Digital.   “It’s an opportunity to further improve the quality process we’ve implemented at hospitals across the province.”

The new AI software searches a database of more than 550,000 x-rays – including 30,000 cases of collapsed lungs – for those most similar to a new patient’s x-rays.

If the known condition in the majority of the similar x-rays is collapsed lung, the AI recommends collapsed lung as the diagnosis in the new case.

“There is no question systems like this will be in place in hospitals within the next two years,” Tizhoosh said. “People are pushing for it and the technology is there.”

The system could offer radiologists a “computational second opinion” in Coral, or be used to prioritize the x-rays busy specialists look at first, reducing treatment delays that also put patients at risk.

In addition to pneumothorax, there are plans to apply the core AI search system to numerous other conditions –  including pneumonia and chronic obstructive pulmonary disease (COPD) – that are diagnosed using chest x-rays.

Researchers will present their work on chest x-rays and pneumothorax at the Vector Institute’s Evolution of Deep Learning Symposium in Toronto this week.

:: https://uwaterloo.ca/news/news/ai-system-more-accurately-identifies-collapsed-lungs-using

:: Watch the video

Searching Is Intelligence

Posted: 2019/08/19 by Admin

August 20-22, 2019 Washington DC. Professor Hamid Tizhoosh will deliver a talk on AI Solutions for Medical Imaging at the NextGen Dx Summit in Washington DC. Image search has emerged as a very promising application of artificial intelligence. He will talk about different challenges of digital pathology and what AI algorithms could offer. Whereas most works focus on classification and segmentation, this talk will attempt to shed light on challenges and opportunities of image search for retrieving useful information from large archives of histopathology images.

Conference at a glance

Conference Brochure

Vector Institute funds Pathfinder Project at Kimia Lab

Posted: 2019/07/16 by Admin

The Vector Institute, an independent, not-for-profit research institute focused on leading-edge machine learning, announced the second in its series of Pathfinder Projects to implement Artificial Intelligence (AI) in the health sector.

The second Pathfinder Project, performed in partnership with the University Health Network (UHN) and the University of Waterloo will enhance radiology diagnoses with AI.

Coral Review, a software solution developed at UHN, is a peer learning tool used by clinicians in diagnostic imaging to support continuous quality improvement of radiologist practice. Using an algorithm developed by Dr. H.R. Tizhoosh, Director of the Laboratory for Knowledge Inference in Medical Image Analysis (Kimia Lab) at UWaterloo and a Faculty Affiliate at the Vector Institute, an AI-enabled Coral Review would scan through thousands of existing medical images (i.e., x-rays) for ones similar to a patient’s and recommend a diagnosis to the attending physician.

“Coral Review currently enables anonymous peer reviews of medical imaging diagnoses. However, it is limited by the availability of physicians who perform the review or ‘second opinion’,” says Leon Goonaratne, Senior Director of Information Technology, UHN. “An AI-enabled peer review solution has the ability to provide the physician with more information when they perform the review, including the identification of images corresponding to rare or difficult to see cases”.

Pathfinder Projects are small-scale efforts designed to produce results in 12 to 18 months that guide future research and technology adoption. With technical and resource support from the Vector Institute, the projects each bring together a multidisciplinary research team to tackle an important health care problem or opportunity using machine learning and AI more broadly. Each project was chosen for its potential to help identify a “path” through which world-class machine learning research can be translated into widespread benefits for patients.

Dr. H. R. Tizhoosh and his team have worked at the nexus of health care and artificial intelligence (AI) for over a quarter century. Yet, only now is the world beginning to see the fruits of that labour. “In spite of the progress we’ve made,” he says, “we’re at the very beginning if we want to bring the technology into hospitals.”

Director of Kimia Lab at the University of Waterloo, Dr. Tizhoosh will be at the forefront of this important shift as he seeks to enhance University Health Network’s (UHN) medical imaging peer review system, Coral Review. It is the second of the Vector Institute’s Pathfinder Projects, which bring together multidisciplinary research teams to tackle important health care problems using machine learning.

Developed at UHN, Coral Review has been implemented at a number of hospitals across Ontario. Designed to bring focus to quality and education within medical imaging departments, the solution enables an anonymous peer review of a medical imaging diagnosis, as well as image quality.

“Coral Review has enabled a program of quality and education for many hospitals,” says Leon Goonaratne, Senior Director of Information Technology, UHN. “While this peer review process is helping identify and facilitate many learning and coaching opportunities across the province, we believe artificial intelligence is the next step to making the solution even more effective”.

To bring more regularity and efficiency into the system, Dr. Tizhoosh and his team are training a machine learning algorithm with a mixture of public and private data set of over 200,000 anonymized medical images. Once trained, the AI-enhanced Coral Review application would find similar looking images from past cases and offer suggested diagnoses, while leaving the final decision to doctors.

“It’s AI deployed in a slightly different way,” says Dr. Tizhoosh. “It allows the radiologist making the diagnosis to benefit from the knowledge of thousands of diagnoses made by other clinicians. That’s very different from making a diagnosis from scratch.”

The teams at UHN and Kimia Lab are starting relatively small, focusing on chest x-rays and specifically looking at pneumothorax, or collapsed lungs. The condition is a technical challenge for radiologists and a practical one for doctors; certain types can be difficult to see on an x-ray and a collapsed lung is both painful and potentially fatal. Small collapses pose a particularly significant challenge. “Doctors can miss small collapses in 40 percent of cases because you just can’t see it,” says Dr. Tizhoosh.

As it currently stands, their algorithm has about a 70 percent accuracy rate. But with technology and resources support from Vector they will fine-tune it over the next year and hope to push that rate above 90 percent before incorporating it into the existing system. Dr. Tizhoosh also hopes to expand the project’s scope beyond pneumothorax. “Long term, we want to add a long list of problems that we automatically check,” he says. “We want to find more difficult problems and work on a larger scale in the radiology domain.”

Once implemented, the system will be the first of its kind: an AI-enabled diagnostic tool for medical images based on image retrieval. “Working with hospitals to implement AI in medical imaging is the most thrilling thing I have ever done in my career,” Dr. Tizhoosh enthuses. “I want to look back and say, ‘this is what I did as a computer scientist.’ It’s a very exciting time.”

“Pneumothorax is a life-threatening emergency. It is typically detected by radiologists reviewing chest X-ray, however, long awaiting worklists will defer treatment”, says Dr. Antonio Szeto, a PDF at Kimia Lab, “Thus, an A.I. system automatically prioritizing X-rays with Pneumothorax for radiologists will benefit patients by reducing time to treatment. Supported by UHN and Vector Institute, we are now training an A.I. system based on Deep Neural Network from more than 500,000 chest X-ray images. It is an exciting project that potentially helps to save lives!”

Can AI Agents be Ethical?

Posted: 2019/07/12 by Admin

Keynote at the 14th Annual London Imaging Discovery Day 2019

London, Ontario, June 12, 2019. Dr. Hamid Tizhoosh, the Director of KIMIA Lab, delivered the keynote at the 14th Annual London Imaging Discovery Day, 2019 on the topic of Artificial Intelligence and ethics.  In his talk, he reviewed the most pressing ethical issues with the deployment of the AI technologies in medical imaging and inquired into the long-term significance of education to develop a better understanding of ethics and intelligence.

This event showcases innovations in research, education and quality of patient care presented by Medical Imaging Scientists, Clinicians, Technologists, Nurses and Administrators; both learners and trainees from London, Ontario and the surrounding areas.

Date and Time:

Wednesday, June 12, 2019
8:00 am – 5:30 pm

Location:
Darryl J. King Student Life Centre

King’s University College

Conference Co-Chairs:
Dr. Narinder Paul
Dr. Aaron Fenster

The event website: https://www.schulich.uwo.ca/medimagin…

Watch the video: Can AI Agents be Ethical? (Youtube)

Artificial Intelligence (AI) in Cancer Imaging: Bridging the Gap between Pathologist and Algorithm

Posted: 2019/06/24 by Admin

JOIN us for free, half-day mini-symposium on Artificial Intelligence (AI) in cancer imaging.

OBJECTIVES

The audience will gain knowledge on intelligent digital imaging workflows, machine learning (ML) tools, and artificial intelligence (AI) analysis that can assist pathologists and support researchers in integrating multilayered image data and machine learning algorithms into cancer diagnostic decision-making.

EVENT DETAILS

Our keynote speakers include Dr. Hamid Tizhoosh, Professor, University of Waterloo, Ontario, and Director of the Knowledge Inference in Medical Image Analysis (KIMIA) Lab; Dr. Phedias Diamandis, Neuropathologist and Clinician Scientist at UHN and Princess Margaret Cancer Centre, and; Dr. Trevor McKee, STTARR imaging facilities, UHN.

The symposium will provide opportunities for researchers, students, and health professionals to:

• meet experts from digital imaging, machine learning, and pathology;
• participate in discussion on how future imaging demands can be met via integration of pathology, imaging and bioinformatics;
• learn more on how AI and ML tools can be conceptualized, synergized, and used in image based clinical tasks for maximizing the pathological image data output.

Date And Time
Fri, June 21, 2019 1:00 PM – 4:30 PM EDT Location: Ontario Institute for Cancer Research 661 University Avenue West Tower, Suite 510 Toronto, ON M5G0A3

Watch the video

Dr. Tizhooshis a Professor in the Faculty of Engineering at University of Waterloo. He is a Sirector of the Knowledge Inference in Medical Image Analysis Lab in the Engineering Faculty at the University of Waterloo. He is also a member of Waterloo AI Institute, and a faculty affiliate to the Vector Institute. His research activities include artificial intelligence, computer vision and medical imaging. He has developed algorithms for medical image filtering, segmentation and search. He is the author of many books and more than 150 scientific articles. Dr. Tizhoosh has extensive experience in working with industry and holds several patents. He is a member of the advisory board for Huron Digital Pathology, Canada.

Dr. Phedias Diamandis is a Neuropathologist and Clinician Scientist at University Health Network’s (UHN) Princess Margaret Cancer Centre. His research focuses on using chemical biology, deep learning and mass spectrometry-based proteomics to resolve phenotype-level heterogeneity in brain glioblastomas. His team is utilizing artificial intelligence and mass spectrometry to define global morphometric and proteomic patterns defining normal development, health maintenance and disease. His group applies machine learning approaches to interrogate datasets and resolve inter- and intra-patient molecular and phenotypic heterogeneity. These machine learning tools can guide more focused validation studies into mechanisms driving neurological disorders.

Dr. Trevor McKee has a PhD from MIT in biological engineering and has 20 years of experience in preclinical imaging, including the development of new algorithms for image analysis. At the STTARR Imaging Facility, Dr. McKee leads a team of algorithm developers to provide image analysis as a service to academic laboratories and pharmaceutical companies, including ongoing analysis for clinical trial specimens. Dr. McKee has worked on developing relationships with pharmaceutical companies to help bridge the translational divide and bring ideas from basic science, through translation in preclinical models at STTARR, and through clinical trials for drugs and imaging agents.

AGENDA

1-2 p.m. How to go digital in pathology

Dr. H. Tizhoosh, KIMIA Lab, University of Waterloo

Introduction to the Digital pathology, a rapidly evolving and essential technology, with specific support for tissue-based research, drug development and the practice of human pathology.

2–2:15 p.m. Artificial Intelligence (AI) algorithms in digital pathology: an overview

Dr. Morteza Babaie and Amir Safarpour, KIMIA Lab, University of Waterloo

Foundations and approaches in developing computerized algorithms for high dimensional data and image analysis for predicting disease outcome. How AI uses algorithms to represent data, classify data, and search for similar instances, either in supervised or unsupervised approaches.

2:15–2:30 p.m. Image search and diagnosis: a first validation using TCGA data

Shivam Kalra, KIMIA Lab, University of Waterloo

An overview, strategies and applications for working on deep networks, metric learning, autoencoders, and searching in large archives of pathology images.

2:30–3:30 p.m. Understanding Machine Engineered Reasoning in Pathology

Dr. Phedias Diamandis, MD, PhD, FRCPC

A pathologist perspective on integration of artificial intelligence and machine learning into diagnostic pathology. Examples of how computer-aided image analysis can be used in various tasks in cancer imaging, e.g. detection, diagnosis, prognosis, and response to therapy. Learn how digital tools can be applied to resolve phenotypic heterogeneity in different glioblastoma niches, empower data mining with patient characteristics to build novel predictive indicators for tumor detection, monitoring and therapy.

3:30–4:30 p.m. To use AI or not? Machine Learning in Practice: clinical trial and translational research applications

Dr. Trevor McKee, PhD, STTARR Imaging facilities

Principles underlying the development of algorithms for the segmentation analysis of histopathology images. Examples of how semi-automated cell-counting strategies on single and multiplexed stained tissue sections can be used for obtaining information on cell markers in relation to cell phenotypes and spatial arrangements. Examples from quantitative analysis on clinical trial specimens will be used to illustrate the current application of machine learning methods in a high-throughput core facility environment.

University of Waterloo’s Kimia Lab and Huron Digital Pathology to Participate in $126M Industry Consortium Led by Sunnybrook Research Institute

Posted: 2019/06/03 by Admin

Waterloo, Canada, June 03, 2019 –(PR.com)– University of Waterloo’s Kimia Lab announced today that it will participate in the $126 million Industry Consortium for Image Guided Therapy (ICIGT) led by the Sunnybrook Research Institute, with investment partnership from the Canadian government.
The Canadian government, through its Strategic Innovation Fund, will invest up to $49 million to support the ICIGT initiative, which, in addition to Kimia Lab and Huron Digital Pathology, consists of more than 70 partners from industry, academia, government organizations and not-for-profits. The consortium’s mandate is to accelerate the application of artificial intelligence and machine learning technologies to deliver better health outcomes, faster diagnoses and safer treatments that minimize side effects and the length of hospital stays.
Kimia and Huron’s project within ICIGT aims to develop intelligent algorithms for consensus building and auto-reporting in digital pathology to improve the speed, cost and accuracy of diagnosis. Huron, in technical partnership with the Kimia Lab, recently introduced the world’s first image search engine that connects pathologists to the vast knowledge contained in the world’s pathology reports.
“This is a historic opportunity to initiate a major change in diagnostic pathology,” says professor Hamid Tizhoosh, Director of Kimia Lab at University of Waterloo. “The AI-driven auto-reporting will be the main output of the project enabling diagnostic consensus by accessing large archives of histopathology images and learning from evidently diagnosed cases of the past.”
“With this project we will develop and bring to market novel technology that addresses a severe shortage of pathologists in Canada and around the world,” adds Patrick Myles, CEO of Huron Digital Pathology. “Together with our fellow ICIGT members, and with support from the Canadian government, we are further positioning Canada as a world leader in leading-edge medical technologies.”
About Kimia Lab
The Laboratory for Knowledge Inference in Medical Image Analysis (short Kimia Lab) is a research group hosted at the Faculty of Engineering, University of Waterloo, On, Canada. Kimia Lab, established in 2013, is a member of Waterloo Artificial Intelligence Institute and conducts research at the forefront of mass image data in medical archives using machine learning schemes with ultimate goal of extracting information that cannot only support a more speedy and accurate diagnosis and treatment of many diseases but also establish new quality assurance based on mining of existing evidence. The lab trains graduate and undergraduate students and annually hosts international visiting scholars. Professor Hamid Tizhoosh, Kimia Lab’s director, is an expert in medical image analysis who has been working on different aspects of artificial intelligence since 1993. He is a faculty affiliate to the Vector Institute.
About Huron Digital Pathology
Based in St. Jacobs, ON, we are on a mission to transform glass slides into shareable knowledge. Our Scan, Index, and Search solution for pathology combines award-winning whole slide imaging hardware with powerful image search technology to connect pathologists, researchers and educators with the expertise of their colleagues to help speed up diagnosis and accelerate disease research. www.hurondigitalpathology.com

Kimia Lab’s Director Delivers Keynote at the 2019 Niagara Investment Summit

Posted: 2019/03/01 by Admin

Waterloo, Ontario, Canada, February 8, 2019 – Kimia Lab’s director, Dr. Hamid R. Tizhoosh will deliver a keynote at the 2019 OBIO Investment Summit on February 21, 2019. The talk is on “Searching is Intelligence: AI Solutions for Medical Imaging” and will explore the potentials of search technologies in medical imaging.
Summit Highlights

• Presentations from Canada’s most innovative health science companies, across three streams: AI in Health, New Frontiers in Medicine, Digital Health and Diagnostics
• One-on-one meetings between investors (venture capital and institutional) and companies
• Keynote presentations from world-leading scientific and business experts
• Panel discussions on cutting-edge topics, featuring leading industry voices addressing critical commercial, scientific, and market access issues
• Exclusive networking dinners for investors and industry CEOs

Waterloo Artificial Intelligence Institute – Inaugural Talk Artificial Intelligence – History, Opportunities and Challenges

Posted: 2018/09/19 by Admin

Waterloo, Ontario, Canada, September 19, 2018 – Waterloo Artificial intelligence Institute will start its Seminar Series this Fall with a talk titled “Artificial Intelligence – History, Opportunities and Challenges”. The talk will be delivered by the Kimia Lab’s director professor Tizhoosh.
Date: Monday, September 24
Time: 3:00-5:00 pm
Location: University of Waterloo, Davis Centre 1302
About the Talk – The history of artificial intelligence (AI) contains several ebbs and flows and is marked by many colorful personalities. We review major milestones in the development of machine learning, starting from principal component analysis to deep networks, and point to a multitude of pivotal developments that have strongly contributed to drawing the historical path of AI. From audacious promise of designing a GPS, General Problem Solver, to the first major “AI Winter”, and from modest success of artificial neural networks in early 90s to the major shift of paradigm in mid 2000s through deep learning, we attempt to understand how AI has reached its today’s position surrounded by enthusiasm and high expectations. We look at major opportunities that AI has created today, from natural language processing to computer vision applications, as well as reviewing the challenges that the AI community is facing today, from black-box behavior to adversarial attacks.
About the Speaker – Hamid R. Tizhoosh is the director of Kimia Lab (Laboratory for Knowledge Inference in Medical Image Analysis) in the Faculty of Engineering at University of Waterloo. Before he joined the University of Waterloo in 2001, he was a research associate at the Knowledge and Intelligence Systems Laboratory at the University of Toronto where he worked on AI methods such as reinforcement learning. Since 1993, his research activities encompass artificial intelligence, computer vision and medical imaging. He has developed algorithms for medical image filtering, segmentation and search. As well, he has introduced the “Opposition-based Learning”. He is the author of two books, 14 book chapters, and more than 140 journal and conference papers. He has also filed 5 patents in collaboration with WatCo (Waterloo Commercialization Office). Dr. Tizhoosh has extensive industrial experience and has worked with numerous companies such as Management of Intelligent Technologies GmbH (Aachen, Germany), Image Processing Systems Inc. (Markham-based company acquired by Photon Dynamics Inc. (San Jose, CA)), and Medipattern Corporation (Toronto). Additionally, Dr. Tizhoosh has more than 10 years of experience in commercialization and start-ups. In 2007, he started Segasist Technologies, a start-up that developed image segmentation software for radiation oncology. The company raised more than $2M under his leadership, both as CEO and CTO. Presently, he is the AI advisor of Huron Digital Pathology, St. Jacobs, ON.

Partners employing AI to help speed up the diagnosis phase

Posted: 2018/06/29 by Admin

Huron Digital Pathology and Grand River Hospital are part of UW study to improve analysis of biopsies
BY  FAISAL ALI, June 28, 2018
Waiting for the results of a biopsy can be an harrowing experience. The anxiety it creates for the patient, as they wait for the results on a suspect growth or damaged tissue, can even lead to sickness, through a phenomenon known as biopsy stress.
The challenges, however, of obtaining a fast diagnosis and turnaround are immense, not in the least because a sample can require the expert vision of a number of highly trained doctors to successfully diagnose. The more doctors reviewing a case, the more accurate the diagnosis, but the practicality of getting five or ten – or, better yet, a hundred – pathologists to review the samples of every single patient at a hospital, and form a single consensus-based diagnosis, are of course nil.
Artificial intelligence may provide an answer to the problem, or at least provide tools to find one, says Hamid Tizhoosh, director of KIMIA Labs at the University of Waterloo’s Artificial Intelligence Institute.
Tizhoosh envisions a novel approach that is “pathologist-centric,” and has partnered up with several organizations, including St. Jacobs-based business Huron Digital Pathology as well as the Grand River Hospital, to bring the idea to life. The project has also received the backing of the province through a $3.1-million grant from the Ontario Research Fund.

The Future of Diagnosis: Learning to Recognize Similar Images in Digital Pathology Speaker: Hamid Tizhoosh, Professor

Posted: 2018/06/28 by Admin

KIMIA Lab (Laboratory for Knowledge Inference in Medical Image Analysis)Faculty of Engineering, University of Waterloo

Large archives of digital scans in pathology are slowly becoming a reality. The amount of information in such archives is not only overwhelming but also, not easily accessible. Fast and reliable search engines, customized for histopathology to perform content-based image retrieval, are urgently needed for a more efficient and informed decision making.

While the mainstream AI is working on classification-oriented framework to make decisions, on behalf of medical/clinical experts, the retrieval approach, in contrast, does not seek to replace the human expert but rather offer assistance by tapping into the collective wisdom of evidently diagnosed cases from the past. Through an ensemble approach, KIMIA Lab at University of Waterloo, offers search engine prototype that exploits the strengths of both handcrafted and deep features for image characterization. The idea of contentbased “barcodes” is subsequently used to accelerate the retrieval process.

Date: Tuesday, July 31, 2018

Time: 3:00 – 4:00 pm

Location: BR 5-20/21, OICR, West Tower, MaRS

Event Organizers:Vanya Peltekova, PhD., Lead, BioLab, OICR

Shivam Kalra receives Governor General Gold Medal

Posted: 2018/06/16 by Admin

June 16, 2018 –  Shivam Kalra, a PhD student at Kimia Lab, has received the Governor General Gold Medal for his academic excellence during his MSc. studies.
“For more than 140 years, the Governor General’s Academic Medals have recognized the outstanding scholastic achievements of students in Canada. They are awarded to the student graduating with the highest average from a high school, as well as from approved college or university programs. Pierre Trudeau, Tommy Douglas, Kim Campbell, Robert Bourassa, Robert Stanfield and Gabrielle Roy are just some of the more than 50 000 people who have received the Governor General’s Academic Medal as the start of a life of accomplishment.” [URL]
Shivam completed his MASc. studies from 2016 to 2018 under the supervision of Profs. Hamid R. Tizhoosh & Shahryar Rahnamayan. His thesis title was “Content-Based Image Retrieval of Gigapixel Histopathology Scans: CNNs versus
LBP versus Bag of Visual Words”.
Shivam started his PhD at Kimia Lab in May 2018. He has been recognized with multiple awards (MITACS, NSERC). He has already published several scientifc works.

Artificial Intelligence and Machine Learning in Medical Imaging

Posted: 2018/06/06 by Admin

June 6, 2018 – Insights from industry: Dr. Hamid R. TizhooshProfessor Faculty of EngineeringUniversity of Waterloo – An interview with Professor Hamid Tizhoosh, conducted by James Ives

Please give an overview of the past research into machine learning and artificial intelligence in medical imaging. What are we currently able to do with this research?

The two major tasks in medical imaging that appear to be naturally predestined to be solved with AI algorithms are segmentation and classification. Most of techniques used in medical imaging were conventional image processing, or more widely formulated computer vision algorithms.
One can find many works with artificial neural networks, the backbone of deep learning. However, most works were focused on conventional computer vision which focused, and still does, on “handcrafted” features, techniques that were the results of manual design to extract useful and differentiating information from medical images.
Some progress was visible in the late 90s and early 2000s (for instance, the SIFT method in 1999, or visual dictionaries in early 2000s) but there were no breakthroughs. However, techniques like clustering and classification were in use with moderate success.
K-means (an old clustering method), support vector machines (SVM), probabilistic schemes, and decisions trees and their extended version ‘random forests’ were among successful approaches.  But artificial neural networks continued to fall short of expectations not just in medical imaging, but in computer vision in general.
Shallow networks (consisting of a few layers of artificial neurons) could not solve difficult problems and deep networks (consisting of many layers of artificial neurons) could not be trained because they were too big. By the mid 2000s there was theoretical progress in this field with the first major success stories in early 2010s on large datasets like ImageNet.
Now suddenly, it was possible to recognise cats and cars in an image, perform facial recognition and automatically label images with a caption describing its content. The investigations of applications of these powerful AI methods in medical imaging has started in the past 3-4 years and is in its infancy but promising results have been reported here and there.

What applications are there for machine learning and artificial intelligence in medical imaging?

Based on recent publications, it seems that the focus of many researchers is on diagnosis, mainly cancer diagnosis, where the output of the AI software is often a “yes/no” decision for malignant/benign, respectively.
The other stream is working on segmenting (marking) specific parts of the images, again with the main attention of many works being on cancer diagnosis and analysis, but also for treatment planning and monitoring.
However, there is much more that AI can offer to medical imaging. Looking at its potentials for radiogenomics, auto-captioning of medical images, recognition of highly non-linear patterns in large datasets, and quantification and visualization of extremely complex image content, are just some examples. We are at the very beginning of an exciting path with many bifurcations.
:: Read the full interview on News Medical Website

$3.7M Research Grant Awarded to Artificial Intelligence Project for Medical Imaging

Posted: 2018/05/14 by Admin

Waterloo, Ontario, Canada, May 14, 2018 – University of Waterloo’s Kimia Lab announced today that its AI project for digital pathology has been awarded a grant by the Ontario Research Fund – Research Excellence program (ORF-RE).

The project aims to develop an intelligent search engine for digital pathology that can retrieve relevant cases from large archives, auto-caption the images, and facilitate consensus building.

“Digital pathology has opened new horizons in medical diagnosis” says Professor Hamid Tizhoosh, the director of Kimia Lab and the Principal Investigator of the project. “At the same time, we have been witnessing the rise of artificial intelligence technologies in recent years that could be applied to discover and exploit the collective wisdom in the big image data”. The project, titled “Computational Peer Review through Identification and Captioning of Gigapixel Digital Pathology Scans” is entirely focused on using, fine-tuning and designing AI algorithms for whole-slide imaging.

The Ontario government will fund the 5-year project with a grant in amount of $3.2M. Huron Digital Pathology (St. Jacobs, ON, Canada), as the industrial partner of Kimia Lab, will contribute $500k to the project. The company is the only Canadian manufacturer of digital scanners for pathology. Four professors from the University of Waterloo (Mark Crowley, Ali Ghodsi, Oleg Mikhailovich, and Hamid Tizhoosh), together with the machine learning group at the University of Guelph led by professor Graham Taylor (Vector Institute), and professor Shahryar Rahnamayan (UOIT) will collaborate with three hospitals to design and test an advanced search engine for large pathology archives. Grand River Hospital (Kitchener, ON), Southlake Regional Health Centre (Newmarket, ON) and University of Pittsburgh Medical Center (PA, USA) will not only provide data but also validate the results of the project.

“We regard this as an exciting and historic opportunity to contribute to the improvement of the healthcare system in such a sensitive and significant field as pathology” adds Tizhoosh, “specially at this point of time when the research in artificial intelligence has started to yield practical results.”

Huron Digital Pathology Collaborates with UPMC on Artificial Intelligence-based Image Search, Presents Pathologist-centric Approach to Image Retrieval at Pathology Informatics Summit in Pittsburgh

Posted: 2018/05/09 by Admin

Huron Digital Pathology announced today that Dr. Hamid Tizhoosh, Director of the KIMIA Lab at the University of Waterloo and Huron advisory board member, will present a talk entitled “Faster, Better, More Reliable than Deep Features: A Projection-Based, Pathologist-Centric Approach to Identification of Histopathology Images” at the Pathology Informatics Summit on May 22, 2018 in Pittsburgh, PA.
The company also announced its research collaboration with the University of Pittsburgh Medical Center (UPMC) and digital pathology pioneer, Dr. Liron Pantanowitz, Professor of Pathology & Biomedical Informatics and Director of Pathology Informatics at UPMC Shadyside, in the field of content-based image retrieval using artificial intelligence.
Dr. Tizhoosh’s abstract presentation at the Pathology Informatics Summit will be the first public introduction to Huron’s technology for identification and retrieval of histopathology images. He will describe a unique approach to image search that places the pathologist in the center of decision making, a key ingredient to driving adoption of artificial intelligence techniques in pathology. The abstract presentation will take place at 8:30am on Tuesday, May 22nd.
“Huron’s whole slide scanners generate petabytes of image data each year, so it made sense for us to develop a simple, yet powerful way to search histopathology images and access underlying knowledge,” commented Patrick Myles, CEO of Huron Digital Pathology. “We are actively validating our technology through collaboration with leading clinical, research and academic institutions as well as key industry partners. We are particularly thrilled to be working with Dr. Pantanowitz, who is providing us with valuable feedback and access to a rich digital pathology archive.”
Dr. Liron Pantanowitz, Professor of Pathology & Biomedical Informatics and Director of Pathology Informatics at UPMC Shadyside, commented “Many of us have witnessed the value of reverse image lookup using Google Image Search. Unfortunately, such content-based image retrieval tools are not readily available for clinical use in pathology. However, it is hoped that future image algorithms, such as the technology Huron are developing today with UPMC and the University of Waterloo, will likely mature to the point of playing an important role in computer assisted diagnosis for anatomic pathology practice.”