About
I am a software engineer at Google working on Vertex AI Prediction service. Prior to…
Activity
-
我们的海外博士项目开始招聘啦！接受2024年博士应届生以及有工作经验的海外博士人才，希望人选有理工科相关专业背景例如计算机软件，人工智能，机器人，机械工程，激光等等～对武汉国企工作感兴趣的小伙伴请联系我～
我们的海外博士项目开始招聘啦！接受2024年博士应届生以及有工作经验的海外博士人才，希望人选有理工科相关专业背景例如计算机软件，人工智能，机器人，机械工程，激光等等～对武汉国企工作感兴趣的小伙伴请联系我～
Liked by Chun-Hsiang Wang
-
At Databricks, we're committed to scaling with you as your organization grows. Learn about our identity and access management best practices and new…
At Databricks, we're committed to scaling with you as your organization grows. Learn about our identity and access management best practices and new…
Liked by Chun-Hsiang Wang
-
tl;dr: I’ve joined Open AI and am hiring SREs. I’m happy to announce that I’ve started a role at Open AI (makers of ChatGPT). I’ll be working on…
tl;dr: I’ve joined Open AI and am hiring SREs. I’m happy to announce that I’ve started a role at Open AI (makers of ChatGPT). I’ll be working on…
Liked by Chun-Hsiang Wang
Experience
-
-
-
-
-
-
-
Education
-
-
-
-
Publications
-
UGSD: User Generated Sentiment Dictionaries from Online Customer Reviews
AAAI-19: Thirty-Third AAAI Conference on Artificial Intelligence
Languages
-
Chinese
Native or bilingual proficiency
-
English
-
More activity by Chun-Hsiang
-
So after over 16.5 years at Google, I appear to have been let go via an automated account deactivation at 3am this morning as one of the lucky…
So after over 16.5 years at Google, I appear to have been let go via an automated account deactivation at 3am this morning as one of the lucky…
Liked by Chun-Hsiang Wang
-
So, yeah, me too. After 17 years, I was laid off on Friday, along with way too many other people at Google. I'm doing fine, honestly it was…
So, yeah, me too. After 17 years, I was laid off on Friday, along with way too many other people at Google. I'm doing fine, honestly it was…
Liked by Chun-Hsiang Wang
-
Hello there, from the depths of lurkerdom. I woke up this morning to our Google Nest Hub in a locked state asking to link my account. I don't recall…
Hello there, from the depths of lurkerdom. I woke up this morning to our Google Nest Hub in a locked state asking to link my account. I don't recall…
Liked by Chun-Hsiang Wang
-
I am excited to share that I will be joining Vinson & Elkins' San Francisco office as a 2023 Summer Associate! I very much look forward to learning…
I am excited to share that I will be joining Vinson & Elkins' San Francisco office as a 2023 Summer Associate! I very much look forward to learning…
Liked by Chun-Hsiang Wang
-
I'm thrilled to be selected as one recipient of 2022 Google PhD Fellowship. Thank Google for the generous support during my Ph.D. study.
I'm thrilled to be selected as one recipient of 2022 Google PhD Fellowship. Thank Google for the generous support during my Ph.D. study.
Liked by Chun-Hsiang Wang
-
Personal Update: I'm thrilled to share that I'll be exploring commercial opportunities around the fast.ai ecosystem full-time! I have been working…
Personal Update: I'm thrilled to share that I'll be exploring commercial opportunities around the fast.ai ecosystem full-time! I have been working…
Liked by Chun-Hsiang Wang
-
I am thrilled to share that I will join Yahoo Search Engineering Team as a Software Engineer Intern this summer! As an international student, it is…
I am thrilled to share that I will join Yahoo Search Engineering Team as a Software Engineer Intern this summer! As an international student, it is…
Liked by Chun-Hsiang Wang
-
New blog post by Jeremy Lewi and myself about a new type of way to put machine learning into production: "Data Science Meets Devops: MLOps with…
New blog post by Jeremy Lewi and myself about a new type of way to put machine learning into production: "Data Science Meets Devops: MLOps with…
Liked by Chun-Hsiang Wang
-
Shopee Singapore is hosting a Virtual Recruitment Day on 15-16 August for Backend Engineers to join our Shopee Engineering team. Because of…
Shopee Singapore is hosting a Virtual Recruitment Day on 15-16 August for Backend Engineers to join our Shopee Engineering team. Because of…
Liked by Chun-Hsiang Wang
Other similar profiles
Explore more posts
-
Harsh Sandhu
LLMs don’t “forget” — they run out of KV cache. At 8K context, a 7B model needs ~448MB just to store keys and values (FP16). Scale context or model size, and this quickly becomes the dominant inference bottleneck. Last week, I came across TurboQuant (arXiv:2504.19874) and implemented a TurboQuant-inspired KV cache quantization pipeline in PyTorch to test how far 3-bit compression can go. Short answer: ~5× smaller KV cache, with minimal quality impact (in this setup). What the method does: ⚙️ → Walsh-Hadamard rotation to distribute activation outliers → 3-bit scalar quantization (Beta-fitted codebook) → Optional 1-bit residual error correction (QJL) Results on Qwen2.5 (0.5B → 7B, Apple Silicon): 📊 ✦ ~4.9× KV cache compression ✦ Key cosine similarity > 0.999 (high-fidelity reconstruction) ✦ 7B @ 8K: 448MB → 91MB ✦ 3B perplexity: 8.16 → ~8.74 (estimated, +0.58) Important caveats: ⚠️ This is an offline POC — quantized KV cache is not yet integrated into live inference Perplexity is approximated, not measured end-to-end No real latency/speed gains yet (requires kernel-level integration) One notable edge case: The 1.5B model showed a large perplexity increase despite high attention fidelity. Likely a quantizer fit issue for that model’s activation distribution — not yet confirmed. Why this matters: 💡 KV cache is heavily over-provisioned in FP16. A reliable ~5× reduction enables: Longer context windows More headroom on the same hardware Running larger models locally (especially on unified memory systems) Compression depends on bit-width and head_dim — not model size — so this should extend to larger models as well. Code: https://lnkd.in/gjkMiWbE Paper: arXiv:2504.19874 (https://lnkd.in/gy44wERh) #MachineLearning #LLM #Quantization #PyTorch #AppleSilicon #LocalAI #TurboQuant
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content