Chicago, IL · B.S. Computer Science · UIC 2027

Building the systems underneath the application layer.

Pranay is a Computer Science student at the University of Illinois Chicago focused on backend engineering, LLM infrastructure, distributed systems, and developer tooling.

view_projectsget_in_touch
pranay@chicago:~/portfolio
$
Portrait of Pranay Karakoti

Pranay Karakoti

Back-End AI Engineering Intern @ FlyRank AI

2.06×
batching throughput
60%
lower short-req latency
236ms
p99 scheduling drift
0
lost jobs under failure
0.876
validation PR-AUC
4.3×
retrieval speedup
02

ABOUT.md

I'm Pranay, a computer science student at the University of Illinois Chicago, graduating May 2027 and based in Chicago.

Most of my work sits underneath the application layer: schedulers, allocators, provider seams, inference loops, and the infrastructure that keeps them honest. I care more about what happens at the queue and the cache than at the button.

I tend to learn things by rebuilding them.

microvllm started as a question about how batching actually works. Chronos started as a question about what happens when a worker dies mid-job. Both became real systems only after the naive versions broke in ways I could measure.

So the rule I hold myself to is simple: measure instead of claiming, benchmark the system, test the failure modes, document the limitations, and keep the failed experiments in the repo. Real engineering evidence beats an inflated number.

profile.yaml
name: Pranay Karakoti
location: Chicago, IL
education: University of Illinois Chicago
degree: B.S. Computer Science
graduation: May 2027

focus:
  - LLM serving infrastructure
  - distributed systems
  - backend engineering
  - developer tooling

currently:
role: Back-End AI Engineering Intern
company: FlyRank AI

open_to:
  - 2027 new grad roles
03

Experience

FlyRank AI

Back-End AI Engineering Intern

Remote · Jul 2026 — Present

  • Designed a provider-agnostic LLM integration layer in Python supporting Groq, Gemini, and Ollama, where switching providers is a configuration change rather than an application-code change.
  • Added Pydantic schema validation with single-retry fallback behavior, plus per-request cost telemetry on every call.
  • Held zero failures across a 10-case reliability suite covering malformed and empty model responses.
  • Migrated storage from in-memory state to PostgreSQL with SQLAlchemy without changing external behavior.
  • Containerized the service with Docker so behavior is preserved across environments.
PythonFastAPIPydanticSQLAlchemyPostgreSQLDockerGroqGeminiOllama
04

Projects

project 01

microvllm

A miniature LLM inference server in C++20 with production-style serving techniques.

Linux / macOS
RequestsAdmission ControlSchedulerContinuous BatchingKV Cache AllocatorLlama.cpp
prefix cache → kv blocks
2.06×
throughput @ batch 16
60%
lower short-req latency
9.9s → 4.4s
repeated-prompt runtime
124
tests
25×
CI TSan runs

Continuous batching, chunked prefill, block-based KV-cache allocation, admission control, and prefix sharing. Llama.cpp handles matrix operations; the queue, scheduler, cache accounting, and serving loop are original.

engineering noteBatching raised throughput until the workload became bandwidth-bound — past that point larger batches stopped paying. The benchmark suite reports the ceiling instead of hiding it.

C++20Llama.cppCMakeThreadSanitizerGoogleTest
github.com/Pranay847 — microvllm(opens in a new tab)
project 02

Chronos

Leaderless distributed job and webhook scheduler on Java 21, Spring Boot, MongoDB.

APIMongoDB atomic coordinationWorker poolJob execution
w0×w2×w44,000 jobs · 0 lost · p99 drift 236ms
2 / 5
workers killed in chaos test
4,000
jobs
0
jobs lost
236ms
p99 scheduling drift
172
tests

No leader and no per-instance configuration: scaling is adding containers. Claiming work is one atomic findAndModify query, so two workers can never take the same job. The design starts from worker failure rather than treating it as an edge case.

Java 21Spring BootMongoDBDockerJUnit
github.com/Pranay847 — Chronos(opens in a new tab)
project 03

M.A.C.E.

Monolith Analysis and Clustering Engine — an agentic tool that proposes service boundaries.

Python MonolithDependency ParserNeo4j GraphGraph ClusteringService BoundariesFastAPI ScaffoldsShadow Testing
30–40%
less codebase analysis time
~20%
fewer deployment errors
6
shadow-tested features

Parses a legacy Python monolith into a dependency graph, clusters it in Neo4j, and emits candidate service boundaries with FastAPI scaffolds that are shadow-tested against the original behavior.

PythonLangChainNeo4jFastAPI
github.com/Pranay847 — M.A.C.E.(opens in a new tab)
project 04

Fraud Detection on Imbalanced Data

0.172% positive rate — a project about evaluation methodology, not accuracy.

recall →val PR-AUC 0.876 · test 0.836
0.876
validation PR-AUC
0.836
held-out test PR-AUC
61 / 74
fraud cases detected

Accuracy is deliberately not reported as a headline metric: a classifier that predicts 'not fraud' every time reaches ~99.8% accuracy and catches nothing. Precision-recall area and per-case recall are the metrics that describe the model honestly.

PythonXGBoostPyTorchPandas
github.com/Pranay847 — Fraud Detection on Imbalanced Data(opens in a new tab)
project 05

Generative AI Stock Portfolio Analyzer

Signal model plus retrieval-augmented context behind a Streamlit interface.

Market DataXGBoost SignalRetrievalFAISS / ChromaDBRAG ContextStreamlit UI
4.8s → 1.1s
end-to-end latency
4.3×
pipeline speedup

Concurrent market-data fetching, batched embeddings, and a single batched vector-store write replaced a serial per-document path — the three changes account for the entire latency improvement.

PythonXGBoostFAISSChromaDBStreamlitRAG
github.com/Pranay847 — Generative AI Stock Portfolio Analyzer(opens in a new tab)
project 06

Chrome AI Writing & Translation Extension

On-device proofreading, rewriting, and translation for the Chrome Built-in AI Challenge.

Chrome
SelectionLanguage DetectionBuilt-in AI ModelOn-device Result
200+
active beta users
50+
languages
18%
avg readability gain

Automatic language detection with proofreading, rewriting, and translation across 50+ languages. Text stays on-device — no server required, no request leaves the browser.

TypeScriptJavaScriptManifest V3Chrome Built-in AI
github.com/Pranay847 — Chrome AI Writing & Translation Extension(opens in a new tab)
project 07

Generative Image Editing Tool

Stable Diffusion editing with 17 natural-language transformation types.

Prompt + ImageHash CacheStable DiffusionInpaint / RestyleOutput
25%
faster repeated runs via hash cache

Style transfer, inpainting, background replacement, and restyling. Hash-based result caching short-circuits repeated prompts against the same source image.

PythonStreamlitStable DiffusionHugging Face Inference API
github.com/Pranay847 — Generative Image Editing Tool(opens in a new tab)
project 08

JobTracker

Full-stack application tracker built because the spreadsheet stopped working.

React / ViteExpressSQLite
50+
applications tracked
~50%
less manual tracking time

JWT authentication with bcrypt password hashing and per-stage application tracking. In real personal use, not a demo dataset.

ReactViteExpressSQLiteJWTbcrypt
github.com/Pranay847 — JobTracker(opens in a new tab)
05

Capability Matrix

languages

  • Primary language: inference services, agents, ML pipelines.
  • Low-level memory and pointer work in coursework and allocators.
  • microvllm: C++20 scheduler, KV allocator, batching loop.
  • Chronos: Java 21 + Spring Boot distributed scheduler.
  • Schema design and query tuning on PostgreSQL and SQLite.
  • JobTracker frontend and Chrome extension logic.
  • Manifest V3 Chrome extension with typed messaging.
  • Accessible, responsive interfaces without frameworks.

backend

  • Provider-agnostic LLM API layer at FlyRank AI.
  • Migrated in-memory state to PostgreSQL models.
  • Schema validation for malformed model responses.
  • Express API for JobTracker.
  • JWT auth, bcrypt hashing, REST routes.
  • Versioned endpoints with explicit error contracts.
  • Rejecting bad LLM output before it reaches storage.
  • Per-request cost and latency telemetry.

cloud / devops

  • Deployment of containerized backend services.
  • Managed container hosting for the API layer.
  • Image registry wired into CI builds.
  • Managed PostgreSQL for persistent job state.
  • Artifact and asset storage.
  • Static distribution and caching.
  • Least-privilege roles for deploy pipelines.
  • Keyless GitHub Actions to AWS auth.
  • Provider API keys outside the image.
  • Private networking for database access.
  • Containerized services with parity across environments.
  • CI incl. 25× ThreadSanitizer runs for microvllm.
  • Trunk-based workflow with reviewed PRs.

databases

  • Primary store after migrating off in-memory state.
  • Dependency graph storage and clustering for M.A.C.E.
  • Embedded store for JobTracker.
  • Atomic findAndModify coordination in Chronos.
  • Vector retrieval in the portfolio analyzer.
  • Batched embedding writes for RAG context.
  • Batched writes to cut end-to-end latency 4.3×.

ai / ml

  • Agentic analysis pipeline in M.A.C.E.
  • Stateful multi-step agent orchestration.
  • Retrieval context for market analysis answers.
  • Specialized agents over a shared code graph.
  • Groq, Gemini, and Ollama behind one interface.
  • Baseline models for imbalanced fraud detection.
  • Inpainting and restyling image tool.
  • Structure-preserving image transformations.
  • 0.876 val PR-AUC on 0.172% positive-rate data.
  • Feature engineering and evaluation splits.
  • Image preprocessing and masking.

systems

  • Worker pools verified under ThreadSanitizer.
  • Block-based KV cache allocator.
  • Prefix sharing: 9.9s → 4.4s repeated prompts.
  • Continuous batching with chunked prefill.
  • Leaderless coordination, chaos-tested.
  • Profile, benchmark, then document the ceiling.
06

How I Build

01

Rebuild it

I understand systems by rebuilding the important pieces myself.

02

Measure it

Benchmarks matter more than claims.

03

Break it

Chaos tests and failure cases expose what happy-path demos hide.

04

Document the limits

Every project should explain not only what works, but where it stops working.

QuestionNaive VersionFailureMeasurementIterationSystem
07

Contact

Have a systems problem worth building?

Open to 2027 new grad roles in LLM serving infrastructure, distributed systems, backend engineering, and developer tooling.

pranay846@outlook.com GitHub(opens in a new tab) LinkedIn(opens in a new tab)
Location
Chicago, IL
Phone
+1 (224) 261-5932
08

Send a message

Sent straight to my inbox.