llmprobe MCP server

llmprobe is a small helper program that checks how well your AI provider is doing. It measures things like how long you wait before the answer starts and how fast it streams. It is handy for anyone who depends on an AI service and wants to know early when it slows down or breaks.

What this MCP server does

You ask your AI to check your AI providers. Your AI calls this helper, and the helper sends a small test message to each provider you set up. It times how long the first word takes, how long the whole answer takes, and how many words per second come back. Then it tells you which providers are healthy, slow, or broken.

Click to zoom

What you can do with it

• Check all your configured AI providers at once and see which are healthy
• Test a single model by name without writing a config file
• List the providers and models you have set up
• See the thresholds you set for speed and errors
• Find out which provider is slow before your users complain
• Use it as a gate before deploying, so slow providers block the release

Try asking your AI

• “Check if all my AI providers are healthy right now”
• “Probe just the gpt-4o model and tell me its time to first token”
• “List the providers and models I have configured”
• “Show me the current config with all thresholds”

What it gives back to you

You get back a short health report for each provider and model. It shows time to first token, total latency, tokens per second, and a status like healthy, degraded, or error. In the chat it looks like a small table or a few lines of plain text.

Before you start

Who it's for

People who run or depend on AI services and want a simple way to check that those services are fast and working.