Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add upcoming webinar banner #82

Merged
merged 1 commit into from
Jul 10, 2024
Merged

add upcoming webinar banner #82

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
<h1 align="center">
<a href="https://encord.com/webinar/tti-eval-webinar/">
<img src="https://storage.googleapis.com/docs-media.encord.com/static/img/text-to-image-eval/fint-tune-text-to-image.jpg" alt="tti-eval logo banner">
</a>
</h1>

<div align="center" style="display: flex; justify-content: space-between;">
<div style="flex: 1; padding: 10px;">
<a href="https://colab.research.google.com/github/encord-team/text-to-image-eval/blob/main/notebooks/tti_eval_CLI_Quickstart.ipynb" target="_blank" style="text-decoration:none"><img alt="CLI Quickstart In Colab" src="https://img.shields.io/badge/Try_It_In_Colab-blue?logo=googlecolab&labelColor=555"></a>
Expand Down Expand Up @@ -114,7 +120,6 @@ To interactively explore the animation in a temporary session, use the `--intera
> <img alt="Python Quickstart In Colab" src="https://img.shields.io/badge/Python_Quickstart_In_Colab-blue?logo=googlecolab&labelColor=555">
> </a>


## Some Example Results

One example of where this `tti-eval` is useful is to test different open-source models against different open-source datasets within a specific domain.
Expand All @@ -134,7 +139,7 @@ The models are evaluated against four different medical datasets. Note, Further
### Weighted KNN Accuracy

| Model/Dataset | Alzheimer-MRI | LungCancer4Types | chest-xray-classification | skin-cancer |
|:-----------------|:-------------:|:----------------:|:-------------------------:|:-----------:|
| :--------------- | :-----------: | :--------------: | :-----------------------: | :---------: |
| apple | 0.6777 | 0.6633 | 0.9687 | 0.7985 |
| bioclip | 0.8952 | 0.7800 | 0.9771 | 0.7961 |
| clip | 0.6986 | 0.6867 | 0.9727 | 0.7891 |
Expand All @@ -148,7 +153,7 @@ The models are evaluated against four different medical datasets. Note, Further
### Zero-shot Accuracy

| Model/Dataset | Alzheimer-MRI | LungCancer4Types | chest-xray-classification | skin-cancer |
|:-----------------|:-------------:|:----------------:|:-------------------------:|:-----------:|
| :--------------- | :-----------: | :--------------: | :-----------------------: | :---------: |
| apple | 0.4460 | 0.2367 | 0.7381 | 0.3594 |
| bioclip | 0.3092 | 0.2200 | 0.7356 | 0.0431 |
| clip | 0.4857 | 0.2267 | 0.7381 | 0.1955 |
Expand All @@ -164,7 +169,7 @@ The models are evaluated against four different medical datasets. Note, Further
### Image-to-image Retrieval

| Model/Dataset | Alzheimer-MRI | LungCancer4Types | chest-xray-classification | skin-cancer |
|:-----------------|:-------------:|:----------------:|:-------------------------:|:-----------:|
| :--------------- | :-----------: | :--------------: | :-----------------------: | :---------: |
| apple | 0.4281 | 0.2786 | 0.8835 | 0.6437 |
| bioclip | 0.4535 | 0.3496 | 0.8786 | 0.6278 |
| clip | 0.4247 | 0.2812 | 0.8602 | 0.6347 |
Expand All @@ -180,7 +185,7 @@ The models are evaluated against four different medical datasets. Note, Further
### Linear Probe Accuracy

| Model/Dataset | Alzheimer-MRI | LungCancer4Types | chest-xray-classification | skin-cancer |
|:-----------------|:-------------:|:----------------:|:-------------------------:|:-----------:|
| :--------------- | :-----------: | :--------------: | :-----------------------: | :---------: |
| apple | 0.5482 | 0.5433 | 0.9362 | 0.7662 |
| bioclip | 0.6139 | 0.6600 | 0.9433 | 0.7933 |
| clip | 0.5547 | 0.5700 | 0.9362 | 0.7704 |
Expand All @@ -206,7 +211,7 @@ This repository contains classification datasets sourced from [Hugging Face](htt
> ⚠️ Currently, only image and image groups datasets are supported, with potential for future expansion to include video datasets.

| Dataset Title | Implementation | HF Dataset |
|:--------------------------|:--------------------------------|:-------------------------------------------------------------------------------------|
| :------------------------ | :------------------------------ | :----------------------------------------------------------------------------------- |
| Alzheimer-MRI | [Hugging Face][hf-dataset-impl] | [Falah/Alzheimer_MRI][Falah/Alzheimer_MRI] |
| chest-xray-classification | [Hugging Face][hf-dataset-impl] | [trpakov/chest-xray-classification][trpakov/chest-xray-classification] |
| LungCancer4Types | [Hugging Face][hf-dataset-impl] | [Kabil007/LungCancer4Types][Kabil007/LungCancer4Types] |
Expand Down Expand Up @@ -285,7 +290,7 @@ _TODO_: Some more prose about what's the difference between implementations.
### Hugging Face Models

| Model Title | Implementation | HF Model |
|:-----------------|:------------------------------|:-----------------------------------------------------------------------------------------------|
| :--------------- | :---------------------------- | :--------------------------------------------------------------------------------------------- |
| apple | [OpenCLIP][open-model-impl] | [apple/DFN5B-CLIP-ViT-H-14][apple/DFN5B-CLIP-ViT-H-14] |
| bioclip | [OpenCLIP][open-model-impl] | [imageomics/bioclip][imageomics/bioclip] |
| eva-clip | [OpenCLIP][open-model-impl] | [BAAI/EVA-CLIP-8B-448][BAAI/EVA-CLIP-8B-448] |
Expand All @@ -303,7 +308,7 @@ _TODO_: Some more prose about what's the difference between implementations.
### Locally Trained Models

| Model Title | Implementation | Weights |
|:-------------|:----------------------------------|:--------|
| :----------- | :-------------------------------- | :------ |
| rsicd-encord | [LocalOpenCLIP][local-model-impl] | - |

### Add a Model from a Known Source
Expand Down
Loading