From cddb37c4985f03152ad78823c4be1c44ae3ed48e Mon Sep 17 00:00:00 2001 From: Jun Siang Cheah Date: Fri, 4 Oct 2024 02:46:50 +0100 Subject: [PATCH] reorder free providers in terms of ease of use/limits --- README.md | 220 ++++++++++++++++---------------- src/pull_available_models.py | 237 ++++++++++++++++++++--------------- 2 files changed, 248 insertions(+), 209 deletions(-) diff --git a/README.md b/README.md index bfbc3fa..6f8ad24 100644 --- a/README.md +++ b/README.md @@ -93,48 +93,86 @@ This lists various services that provide free access or credits towards API-base embedding-001 - Google Cloud Vertex AI - Very stringent payment verification for Google Cloud. - Llama 3.1 405B Instruct - Llama 3.1 API Service free during preview.
60 requests/minute - - - Llama 3.1 70B Instruct - Llama 3.1 API Service free during preview.
60 requests/minute - - - Llama 3.1 8B Instruct - Llama 3.1 API Service free during preview.
60 requests/minute - - - Llama 3.2 90B Vision Instruct - Llama 3.2 API Service free during preview.
30 requests/minute - - - Gemini Flash Experimental - Experimental Gemini model.
10 requests/minute - - - Gemini Pro Experimental - - glhf.chat (Free Beta) - Email for API access - Any model on Hugging Face runnable on vLLM and fits on a A100 node (~640GB VRAM), including Llama 3.1 405B at FP8 - - - Cohere - 20 requests/min
1000 requests/month - Command-R - Shared Limit + Lambda Labs (Free Preview) + Free for a limited time + Nous Hermes 3 Llama 3.1 405B (FP8) + - Command-R+ + + Liquid LFM 40B + + Mistral (Le Platforme) + Free tier (Experiment plan) requires opting into data training. + Open and Proprietary Mistral models + 1 request/second
500,000 tokens/minute
1,000,000,000 tokens/month + + Mistral (Codestral) + Currently free to use, monthly subscription based, requires phone number verification. + Codestral + 30 requests/minute
2000 requests/day + HuggingFace Serverless Inference Limited to models smaller than 10GB.
Some popular models are supported even if they exceed 10GB. Various open models 50 requests/hour (with an account) - OVH AI Endpoints (Free Alpha)Token expires every 2 weeks.CodeLlama 13B Instruct12 requests/minute + + SambaNova Cloud + + Llama 3.1 405B + + + + Llama 3.1 70B + + + + Llama 3.1 8B + + + + Llama 3.2 3B + + + + Llama 3.2 1B + + + Cerebras + Waitlist
Free tier restricted to 8K context + Llama 3.1 8B + 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day + + + Llama 3.1 70B + 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day + GitHub ModelsWaitlist
Rate limits dependent on Copilot subscription tierAI21-Jamba-Instruct +Cohere Command R +Cohere Command R+ +Cohere Embed v3 English +Cohere Embed v3 Multilingual +Meta-Llama-3-70B-Instruct +Meta-Llama-3-8B-Instruct +Meta-Llama-3.1-405B-Instruct +Meta-Llama-3.1-70B-Instruct +Meta-Llama-3.1-8B-Instruct +Mistral Large +Mistral Large (2407) +Mistral Nemo +Mistral Small +OpenAI GPT-4o +OpenAI GPT-4o mini +OpenAI Text Embedding 3 (large) +OpenAI Text Embedding 3 (small) +Phi-3-medium instruct (128k) +Phi-3-medium instruct (4k) +Phi-3-mini instruct (128k) +Phi-3-mini instruct (4k) +Phi-3-small instruct (128k) +Phi-3-small instruct (8k) +Phi-3.5-mini instruct (128k) +OVH AI Endpoints (Free Alpha)Token expires every 2 weeks.CodeLlama 13B Instruct12 requests/minute Llama 2 13B Chat12 requests/minute Llama 3 70B Instruct12 requests/minute Llama 3 8B Instruct12 requests/minute @@ -183,86 +221,48 @@ This lists various services that provide free access or credits towards API-base TinyLlama 1.1B Chat v1.0 Una Cybertron 7B v2 (BF16) Zephyr 7B Beta (AWQ) - - Lambda Labs (Free Preview) - Free for a limited time - Nous Hermes 3 Llama 3.1 405B (FP8) - - - - - Liquid LFM 40B - - - Mistral (Le Platforme) - Free tier (Experiment plan) requires opting into data training. - Open and Proprietary Mistral models - 1 request/second
500,000 tokens/minute
1,000,000,000 tokens/month - - Mistral (Codestral) - Currently free to use, monthly subscription based, requires phone number verification. - Codestral - 30 requests/minute
2000 requests/day - - Cerebras - Waitlist
Free tier restricted to 8K context - Llama 3.1 8B - 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day - - - Llama 3.1 70B - 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day - - SambaNova Cloud - - Llama 3.1 405B - - - - Llama 3.1 70B - - - - Llama 3.1 8B - - - - Llama 3.2 3B - - - - Llama 3.2 1B - - GitHub ModelsWaitlist
Rate limits dependent on Copilot subscription tierAI21-Jamba-Instruct -Cohere Command R -Cohere Command R+ -Cohere Embed v3 English -Cohere Embed v3 Multilingual -Meta-Llama-3-70B-Instruct -Meta-Llama-3-8B-Instruct -Meta-Llama-3.1-405B-Instruct -Meta-Llama-3.1-70B-Instruct -Meta-Llama-3.1-8B-Instruct -Mistral Large -Mistral Large (2407) -Mistral Nemo -Mistral Small -OpenAI GPT-4o -OpenAI GPT-4o mini -OpenAI Text Embedding 3 (large) -OpenAI Text Embedding 3 (small) -Phi-3-medium instruct (128k) -Phi-3-medium instruct (4k) -Phi-3-mini instruct (128k) -Phi-3-mini instruct (4k) -Phi-3-small instruct (128k) -Phi-3-small instruct (8k) -Phi-3.5-mini instruct (128k) Together Llama 3.2 11B Vision Instruct Free for 2024 + + Cohere + 20 requests/min
1000 requests/month + Command-R + Shared Limit + + + Command-R+ + + Google Cloud Vertex AI + Very stringent payment verification for Google Cloud. + Llama 3.1 405B Instruct + Llama 3.1 API Service free during preview.
60 requests/minute + + + Llama 3.1 70B Instruct + Llama 3.1 API Service free during preview.
60 requests/minute + + + Llama 3.1 8B Instruct + Llama 3.1 API Service free during preview.
60 requests/minute + + + Llama 3.2 90B Vision Instruct + Llama 3.2 API Service free during preview.
30 requests/minute + + + Gemini Flash Experimental + Experimental Gemini model.
10 requests/minute + + + Gemini Pro Experimental + + glhf.chat (Free Beta) + Email for API access + Any model on Hugging Face runnable on vLLM and fits on a A100 node (~640GB VRAM), including Llama 3.1 405B at FP8 + ## Providers with trial credits diff --git a/src/pull_available_models.py b/src/pull_available_models.py index 6ec5d4e..6b06873 100644 --- a/src/pull_available_models.py +++ b/src/pull_available_models.py @@ -532,10 +532,30 @@ def main(): """ for idx, model in enumerate(groq_models): - table += f"{f'Groq' if idx == 0 else ''}{ f'' if idx == 0 else ''}{model['name']}{get_human_limits(model)}\n" + table += "" + + if idx == 0: + table += f'' + table += 'Groq' + table += '' + table += f'' + + table += f"{model['name']}" + table += f"{get_human_limits(model)}" + table += "\n" for idx, model in enumerate(openrouter_models): - table += f"{f'OpenRouter' if idx == 0 else ''}{ f'' if idx == 0 else ''}{model['name']}{get_human_limits(model)}\n" + table += "" + + if idx == 0: + table += f'' + table += 'OpenRouter' + table += '' + table += f'' + + table += f"{model['name']}" + table += f"{get_human_limits(model)}" + table += "\n" table += f""" Google AI Studio @@ -574,6 +594,122 @@ def main(): embedding-001 """ + + table += """ + Lambda Labs (Free Preview) + Free for a limited time + Nous Hermes 3 Llama 3.1 405B (FP8) + + + + + Liquid LFM 40B + + """ + + table += """ + Mistral (Le Platforme) + Free tier (Experiment plan) requires opting into data training. + Open and Proprietary Mistral models + 1 request/second
500,000 tokens/minute
1,000,000,000 tokens/month + """ + + table += """ + Mistral (Codestral) + Currently free to use, monthly subscription based, requires phone number verification. + Codestral + 30 requests/minute
2000 requests/day + """ + + table += """ + HuggingFace Serverless Inference + Limited to models smaller than 10GB.
Some popular models are supported even if they exceed 10GB. + Various open models + 50 requests/hour (with an account) + """ + + table += """ + SambaNova Cloud + + Llama 3.1 405B + + + + Llama 3.1 70B + + + + Llama 3.1 8B + + + + Llama 3.2 3B + + + + Llama 3.2 1B + + """ + + table += """ + Cerebras + Waitlist
Free tier restricted to 8K context + Llama 3.1 8B + 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day + + + Llama 3.1 70B + 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day + """ + + for idx, model in enumerate(github_models): + table += "" + table += f'GitHub Models' if idx == 0 else '' + table += f'Waitlist
Rate limits dependent on Copilot subscription tier' if idx == 0 else '' + table += f"{model['name']}" + table += "" + table += "\n" + + for idx, model in enumerate(ovh_models): + table += "" + if idx == 0: + table += '' + table += 'OVH AI Endpoints (Free Alpha)' + table += '' + table += 'Token expires every 2 weeks.' + table += f"{model['name']}" + table += f"{get_human_limits(model)}" + table += "\n" + + for idx, model in enumerate(cloudflare_models): + table += "" + if idx == 0: + table += '' + table += 'Cloudflare Workers AI' + table += '' + table += '' + table += '10000 tokens/day' + table += '' + table += f"{model['name']}" + table += "" + table += "\n" + + table += """ + Together + + Llama 3.2 11B Vision Instruct + Free for 2024 + """ + + table += """ + Cohere + 20 requests/min
1000 requests/month + Command-R + Shared Limit + + + Command-R+ + """ table += """ Google Cloud Vertex AI @@ -608,103 +744,6 @@ def main(): """ - table += """ - Cohere - 20 requests/min
1000 requests/month - Command-R - Shared Limit - - - Command-R+ - """ - - table += """ - HuggingFace Serverless Inference - Limited to models smaller than 10GB.
Some popular models are supported even if they exceed 10GB. - Various open models - 50 requests/hour (with an account) - """ - - for idx, model in enumerate(ovh_models): - table += f"{f'OVH AI Endpoints (Free Alpha)' if idx == 0 else ''}{ f'Token expires every 2 weeks.' if idx == 0 else ''}{model['name']}{get_human_limits(model)}\n" - - for idx, model in enumerate(cloudflare_models): - table += f"{f'Cloudflare Workers AI' if idx == 0 else ''}{ f'10000 tokens/day' if idx == 0 else ''}{model['name']}\n" - - table += """ - Lambda Labs (Free Preview) - Free for a limited time - Nous Hermes 3 Llama 3.1 405B (FP8) - - - - - Liquid LFM 40B - - """ - - table += """ - Mistral (Le Platforme) - Free tier (Experiment plan) requires opting into data training. - Open and Proprietary Mistral models - 1 request/second
500,000 tokens/minute
1,000,000,000 tokens/month - """ - - table += """ - Mistral (Codestral) - Currently free to use, monthly subscription based, requires phone number verification. - Codestral - 30 requests/minute
2000 requests/day - """ - - table += """ - Cerebras - Waitlist
Free tier restricted to 8K context - Llama 3.1 8B - 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day - - - Llama 3.1 70B - 30 requests/minute, 60000 tokens/minute
900 requests/hour, 1000000 tokens/hour
14400 requests/day, 1000000 tokens/day - """ - - table += """ - SambaNova Cloud - - Llama 3.1 405B - - - - Llama 3.1 70B - - - - Llama 3.1 8B - - - - Llama 3.2 3B - - - - Llama 3.2 1B - - """ - for idx, model in enumerate(github_models): - table += "" - table += f'GitHub Models' if idx == 0 else '' - table += f'Waitlist
Rate limits dependent on Copilot subscription tier' if idx == 0 else '' - table += f"{model['name']}" - table += "" - table += "\n" - - table += """ - Together - - Llama 3.2 11B Vision Instruct - Free for 2024 - """ - table += "" trial_table = ""