What is TezGrid?
TezGrid is an inference service that routes chat requests to computers owned by people and businesses, rather than relying only on company-owned data-center hardware. It is aimed at app builders looking to serve model responses through an API, as well as computer owners who want to earn from idle machines. The service supports Macs and Windows or Linux computers with NVIDIA, AMD, Intel, or Apple hardware. TezGrid is inviting users to join its early-access list.
TezGrid Features:
Chat API: Apps can send chat requests to TezGrid through a single API address, with streaming responses and existing OpenAI client code supported.
Multi-Platform Hosting: Model jobs can run on Macs, Windows or Linux computers, and systems with NVIDIA, AMD, Intel, or Apple processors.
Request Routing: TezGrid selects a computer using factors including speed, city, trust, and price, and can return the region that served a response.
Retry Handling: If a computer goes offline or a response is cut off, TezGrid can move the job to another working computer.
Trust Controls: Each response identifies the computer's trust level, and apps can set a minimum level for the computers they will accept.
Keys and Limits: The service provides API keys, per-client spending limits, a dashboard for jobs and costs, and protection against charging again for a retried request.
TezGrid Benefits:
Lower Compute Costs: Using already-purchased computers can reduce reliance on data-center capacity, while charges are based on requests rather than idle rental hours.
Less Integration Work: Teams can keep their existing chat code and connect by changing the API address instead of adopting a new library or rewriting an app.
Closer Responses: Routing requests to available computers near users can reduce the distance a response travels, which may help with latency.
Clearer Data Choices: Trust information and minimum-level settings give teams a way to reject some computers, while making the privacy limits visible.
Value From Idle PCs: Computer owners can earn from machines that would otherwise sit unused, and TezGrid shows their share for each completed job.
Use Cases:
App Chat Features: A founder can connect an existing chatbot or conversational app to the API without replacing its current OpenAI-style client code.
Support Ticket Triage: A business can use smaller models on incoming support tickets to sort or tag requests before a person handles them.
Mail Classification: An app can send routine email-sorting jobs to available computers instead of keeping a rented GPU running between requests.
Regional Support Bots: A team serving customers in different regions can route chat requests toward nearby computers when suitable capacity is available.
Monetizing Spare Hardware: A gaming PC owner, Mac user, or office can put an idle computer online to run model requests and receive a share of the charge.

