When a CTO asks how many servers Orchestra needs, our first answer is a question: how many of your people will use it at the busiest hour of the day?

Most quotes for on-premises AI start from the other end. The vendor picks a large model, works out how many GPUs it takes to run, and prices from there. That number describes the model and says little about your organisation. Projects often stall at this point, because the bill arrives before anyone has asked how the system will be used.

Four questions about the work

We start sizing with four questions:

  1. How many people will be signed in on a normal working day?
  2. How many of them will be waiting for an answer at the same moment during the busiest hour?
  3. Will assistants run longer jobs in the background, such as reading a batch of invoices overnight?
  4. How many documents will the system need to read, and how often do they change?

The second question matters most, and its answer is usually far lower than the first. People read an answer, think about it, go to a meeting and come back later. So we size for the busy hour, then confirm the figure with a load test on realistic Arabic questions before go-live.

Decide what a failure should cost you

The next decision is about resilience, and it belongs to the business more than to IT. A starter setup uses the fewest servers. If one of them fails, the assistant stops until it is repaired or restored from backup. A resilient setup adds servers, so the assistant keeps answering at reduced capacity while the faulty one is fixed.

A pilot in one department can usually live with a few hours of downtime. An assistant that finance and HR rely on every day usually cannot. We put both options in the proposal with their prices, so the decision is made in the open.

Your own vendor prices the hardware

NoorAI does not sell hardware. We give you a short specification, and your usual supplier or cloud provider prices it. We ask for quotes on more than one GPU model, because the GPU is the largest single cost and its price and availability change from month to month. The other servers are ordinary machines your team already knows how to run. Equivalent cloud servers in your own country work too, if your rules allow it.

Once the hardware is in place, the cost of using it does not grow with every question. Orchestra is licensed per year with no charge per token, so the budget you approve is the budget you spend.

Why Orchestra needs less hardware

Orchestra needs less hardware because we built the model and the way it is served together, for Arabic. Its tokenizer was designed for Arabic, so a sentence splits into fewer pieces and each answer takes less work. The model also uses only part of itself for each answer, which leaves more memory free for the other conversations running at the same time.

The saving goes beyond the purchase price. Fewer GPUs means less power, less cooling and less rack space. It also means a shorter wait for delivery, and lead times on high-end GPUs can decide whether a project starts this quarter or next year.

Questions to ask any vendor

  • How many people at the busiest hour is this quote sized for, and how was that number worked out?
  • What happens to users if one server fails?
  • Can our own hardware supplier price this specification?
  • What is the minimum number of GPUs, of which model, and why?
  • Has the sizing been checked with a load test on realistic Arabic questions?

If the answer is one large number with no workload behind it, ask how it was reached. For where Orchestra can run, read our comparison of on-premises and cloud deployment.

Related pages