My Lemmy Oracle
  • Communities
  • Create Post
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
cfgaussian@lemmygrad.ml to Technology@lemmygrad.ml · 19 days ago

Alibaba cuts AI GPU use by 82% with new pooling system— up to 9x increase in output lets 213 GPUs perform like 1,192

www.tomshardware.com

external-link
message-square
17
fedilink
  • cross-posted to:
  • [email protected]
54
external-link

Alibaba cuts AI GPU use by 82% with new pooling system— up to 9x increase in output lets 213 GPUs perform like 1,192

www.tomshardware.com

cfgaussian@lemmygrad.ml to Technology@lemmygrad.ml · 19 days ago
message-square
17
fedilink
  • cross-posted to:
  • [email protected]
Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system— up to 9x increase in output lets 213 GPUs perform like 1,192
www.tomshardware.com
external-link
A paper presented at SOSP 2025 details how token-level scheduling helped one GPU serve multiple LLMs, reducing demand from 1,192 to 213 H20s.
  • darkernations@lemmygrad.ml
    link
    fedilink
    arrow-up
    6
    ·
    19 days ago

    Excellent share; thank you

Technology@lemmygrad.ml

technology@lemmygrad.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]

A tech news sub for communists

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 13 users / day
  • 150 users / week
  • 425 users / month
  • 946 users / 6 months
  • 1 local subscriber
  • 1.31K subscribers
  • 1.24K Posts
  • 3.37K Comments
  • Modlog
  • mods:
  • Muad'Dibber@lemmygrad.ml
  • BE: 0.19.5
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org