97¹ú¼Ê

Àë±ðÔËάÄÚÚ§ È«ÓòЭͬÌáЧ Ø­ 97¹ú¼ÊÍøÂçÔËά°ü¹ÜÂÄÀú·ÖÏí»á
Ô¤Ô¼Ö±²¥
ÎÞ¸Ð×¼Èë ÈËÎïͳ¹Ü Ø­ RG-SAM+5.X ÐÂÒ»´ú¸ßУAIÈÏ֤ƽ̨Ðû²¼
Ô¤Ô¼Ö±²¥
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
²úÆ·
< ·µ»ØÖ÷²Ëµ¥
²úÆ·ÖÐÐÄ
²úÆ·
½â¾ö¼Æ»®
< ·µ»ØÖ÷²Ëµ¥
½â¾ö¼Æ»®ÖÐÐÄ
ÐÐÒµ
ºÏ×÷»ï°é
·µ»ØÖ÷²Ëµ¥
Ñ¡ÔñÇøÓò/ÓïÑÔ
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾ 97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

½âÃÜDeepSeek-V3ÍÆÀíÍøÂ磺MoE¼Ü¹¹ÔõÑùÖØ¹¹µÍʱÑÓ¡¢¸ßÍÌÍÂÐèÇó£¿

DeepSeek-V3Ðû²¼Íƶ¯ÂþÑÜÊ½ÍÆÀíÍøÂç¼Ü¹¹Éý¼¶ £¬MoEÄ£×ÓÒýÈë´ó¹æÄ£×¨¼Ò²¢ÐÐͨѶ £¬ÍÆÀíÁ÷Á¿ÌØÕ÷ÏÔÖø×ª±ä £¬Decode½×¶Î¶ÔÍøÂçʱ¶ÈÃô¸Ð¡£ÍøÂçÐè°ü¹ÜµÍʱÑÓÓë¸ßÍÌÍ £¬Í¨¹ý¶ËÍøÐ­Í¬¸ºÔØÆ½ºâÓëÓµÈû¿ØÖÆÊÖÒÕÓÅ»¯ÐÔÄÜ¡£¸ßЧÔËάʵÏÖ¹ÊÕÏ¿ìËÙ¶¨Î»ÓëÓªÒµ¸ß¿ÉÓà £¬µ¥¹ìË«Æ½ÃæÓëShuffle¶àÆ½Ãæ×éÍø¼Æ»®Ôڵͱ¾Ç®ÏÂÖª×ã¸ßÐÔÄÜÍÆÀíÐèÇó £¬Îª´ó¹æÄ£MoEÄ£×Ó°²ÅÅÌṩ½¹µãÍøÂçÖ§³Ö¡£

  • 97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

    Ðû²¼Ê±¼ä£º2025-10-27

  • 97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

    µã»÷Á¿£º

  • 97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

    µãÔÞ£º

·ÖÏíÖÁ

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

ÎÒÏë̸ÂÛ

Ò»¡¢ÍÆÀí³¡¾°ºÍMoEÄ£×ÓÒýÈëÍøÂçÐÂËßÇó

2025ÄêÍ· £¬DeepSeek-V3Ðû²¼ £¬Ñ¸ËÙÒý·¢º£ÄÚÍâµÄÆÕ±é¹Ø×¢ºÍ°²ÅÅÈȳ±¡£×÷Ϊ½¹µã»ù´¡Éèʩ֮һ £¬ÂþÑÜÊ½ÍÆÀíÍøÃæÁÙȫеÄÐèÇó¡£ÕûÌåÀ´¿´ £¬ÍÆÀíÓëѵÁ·µÄÁ÷Á¿²î±ð¡¢MoEÄ£×Ӽܹ¹µÄÒýÈëÒÔ¼°DeepSeek¿ªÔ´ÊÖÒռƻ®µÈ¶àÖØÒòËØ £¬Ó°ÏìÁËÍøÂ罨ÉèµÄÆ«ÏòºÍÒªÇó¡£

¹Å°åŨÃÜÄ£×ÓµÄѵÁ·ÓëÍÆÀíÁ÷Á¿ÖÐ £¬95%ÒÔÉÏΪTensor Parallel£¨TP£©Í¨Ñ¶ £¬Ö÷ÒªÔÚ»úÄڸߴø¿íÓòͨ¹ýall-reduceÍê³É £¬»úÍâµÍ´ø¿íÓò½öÔÚͬºÅ¿¨¼äÖ´ÐеÍÁ÷Á¿µÄÊý¾Ý²¢ÐУ¨DP£©ºÍÁ÷Ë®Ïß²¢ÐУ¨PP£©Í¨Ñ¶¡£¶øDeepSeek½ÓÄɵÄMoE£¨Mixture of Experts£©Ä£×Ӽܹ¹ÏÔÖø¸Ä±äÁËÁ÷Á¿ÌØÕ÷¡£ÑµÁ·ºÍÍÆÀí½×¶Î¾ù²»½ÓÄÉTPͨѶ £¬È¡¶ø´úÖ®µÄÊÇ´ó¹æÄ£×¨¼Ò²¢ÐУ¨EP£©Í¨Ñ¶ £¬ÑµÁ·½×¶ÎEPÁ÷Á¿Õ¼±ÈÁè¼Ý95% £¬ÍÆÀí½×¶ÎÔòµÖ´ï100%¡£EPͨѶ¿çÔ½¶à¸öÆéá«´ø¿íÓò £¬ÇÒ½ÓÄÉall-to-allͨѶģʽ £¬Í¨Ñ¶½á¹¹ÖØ´óÇÒÁ÷Á¿ÖØ´ó £¬¶ÔÍøÂçÐÔÄÜÌá³öÁ˸ü¸ß¡¢¸ü²î±ð»¯µÄÒªÇó¡£

DeepSeekÄ£×Ó²ÎÊý¹æÄ£µÖ´ï6710ÒÚ £¬ÔÚÍÆÀí°²ÅÅÖÐÒýÈëÁËPDÊèÉ¢ºÍ´ó¹æÄ£EP²¢ÐÐ £¬Íƶ¯ÂúѪ°æ¸ßÐÔÄÜÍÆÀí×ßÏòÂþÑÜʽ¡£Ïà±È¹Å°åµ¥»úÍÆÀí £¬ÂþÑÜÊ½ÍÆÀí´øÀ´ÁËÏÔÖø²î±ð £¬Ê¹µÃÍÆÀíÁ÷Á¿Ä£Ê½ÓëÂþÑÜʽѵÁ·¸üΪ¿¿½ü £¬µ«Á½ÕßÔÚÁ÷Á¿ÌØÕ÷ÉÏÒÀÈ»±£´æÏÔ×ÅÇø±ð¡£

ͨѶÁ÷Á¿¿ÉÓÉÒÔϹ«Ê½¹ÀË㣺£¨minibatch¾Þϸ × ÉÏÏÂÎij¤¶È × Òþ²Ø²ãά¶È£©× ½ÚµãÊý × £¨dispatch_alltoallͨѶ´ÎÊý × FP8×Ö½ÚÊý + combine_alltoallͨѶ´ÎÊý × BF16×Ö½ÚÊý£©× GPUÈÏÕæµÄ²ãÊý¡£Ï±íͳ¼ÆÖ÷ÒªEPÁ÷Á¿×÷Ϊ²Î¿¼¡£

×ÜͨѶÁ¿ µ¥´ÎͨѶÁ¿
ѵÁ· 315GB

dispatch£º112MB

combine£º224MB

ÍÆÀíPrefill 57.09GB

dispatch£º168MB

combine£º336MB

ÍÆÀíDecode 1218MB

dispatch£º3.5MB

combine£º7MB

ѵÁ·³¡¾°Á÷Á¿Ä£Ê½Àο¿ÇÒÃ÷È· £¬µ¥´Îµü´ú×ÜÁ÷Á¿¸ß´ï315GB £¬µ¥´ÎEPͨѶÁ÷Á¿Ô¼112MB¡£

ÍÆÀí³¡¾°Á÷Á¿ÊÜÓû§ÊäÈëÓ°Ïì £¬²¨¶¯½Ï´ó¡£Prefill½×¶ÎÒÔ4KÉÏÏÂÎÄ¡¢batch sizeΪ4ÅÌËãÁ÷Á¿¾Þϸ £¬µ¥´Îµü´ú×ÜÁ÷Á¿Ô¼57.09GB £¬µ¥´ÎͨѶÁ÷Á¿ÓëѵÁ·Ïà½ü£»Decode½×¶ÎÒÔ128²¢·¢ÅÌËã £¬µ¥´Îµü´úÁ÷Á¿ÏÔÖø½µµÍÖÁÔ¼1.2GB £¬µ¥´ÎͨѶÁ÷Á¿½öΪ¼¸MB £¬PrefillÓëDecode½×¶ÎÁ÷Á¿²î±ðÏÔ×Å¡£

»ùÓÚÒÔÉÏÈ«ÐÂÇÒÖØ´óµÄÍøÂçÐèÇó £¬ÉîÈëʶ±ðºÍÆÊÎöDeepSeekÍÆÀíÍøÂçµÄÒªº¦ÊÖÒÕ £¬Êǰü¹ÜÍÆÀí¸ßÐÔÄÜ¡¢µÍ±¾Ç®Óë¸ß¿É¿¿ÐÔµÄÒªº¦¡£ÏÂÎÄÎÒÃǽ«´ÓµÍÍøÂçʱÑÓ¡¢¸ßÐ§ÍøÂçÔËάºÍµÍ±¾Ç®×éÍø½Ç¶È £¬Õö¿ªÏÈÈÝDeepSeekÍÆÀíÍøÂçÒªº¦ÊÖÒÕ¡£

¶þ¡¢µÍʱÑÓÍøÂçÖúÁ¦ÍÆÀí¸ßÍÌÍÂ

ƾ֤ÉÏÊöÁ÷Á¿ÆÊÎö £¬Decode½×¶ÎµÄµ¥´ÎͨѶÁ÷Á¿½öΪ3.5MB/7MB¡£ÍŽáDeepSeek¹Ù·½¿ªÔ´Í¨Ñ¶¿âDeepEPµÄÐÔÄÜ £¬Ä¿½ñ³¡¾°ÏÂDecode½×¶ÎµÄdispatchͨѶʱ³¤ÔÚ100usÄÚ £¬combineͨѶʱ³¤ÔÚ200usÄÚ¡£Decode½×¶ÎµÄSLOͨ³£ÒªÇóµÍÓÚ50ms £¬µ«EPͨѶ´ÎÊý¸ß´ï116´Î £¬Ã¿´ÎͨѶ¶¼»áµ¼ÖÂʱÑÓµþ¼Ó £¬Òò´Ë¶ÔÍøÂçʱÑÓÌá³öÁ˺ܸߵÄÒªÇó¡£×ÛÉÏ £¬ÔÚDecode½×¶Î £¬ºÜÉٵĵ¥´ÎͨѶÁ÷Á¿¡¢ºÜ¶ÌµÄͨѶʱ³¤¡¢ºÜ¸ßµÄSLOÒªÇó¶¼¶ÔÍøÂçÌá³öÁ˽ϵ͵ÄʱÑÓÐèÇó¡£

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

H800ÍøÂçʱÑÓ¶ÔDecodeÍÌ͵ÄÓ°Ïì

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

H20ÍøÂçʱÑÓ¶ÔDecodeÍÌ͵ÄÓ°Ïì

ÉÏͼÊǶÔ4K/1KÉÏÏÂÎÄ £¬1KÊä³öµÄDecode³¡¾° £¬ÔÚH800/H20×°±¸Ï £¬ÒÔ128 batch×÷Ϊ³¡¾° £¬¾ÙÐеÄÍøÂçʱÑÓ¶ÔDecodeÍÌÍÂÓ°Ïì·ÂÕæ¡£ÈçͼËùʾ £¬µ±ÍøÂç²à±¬·¢1msµÄʱÑÓÔöÌíʱ £¬ÎÞÂÛÊÇH800ÕÕ¾ÉH20 £¬ÔÚ²î±ðµÄÉÏÏÂÎij¡¾°Ï £¬ÍÌͶ¼»á±¬·¢ÖØ´óÓ°Ïì £¬ÍÌÍÂϽµ·ù¶È¸ß´ï80%×óÓÒ £¬ÏÕЩÒѾ­Ö±½Óµ¼ÖÂÄ¿½ñDecode½Úµã²»¿ÉÓᣵ±ÍøÂçÉϱ¬·¢100usµÄʱÑÓʱ £¬4KÉÏÏÂÎij¡¾°Ï £¬ÍÌÍÂϽµ¿ÉÄִܵï20%+¡£Óɴ˿ɼû £¬Decode½Úµã¶ÔÍøÂçʱÑÓµÄÃô¸Ð¶ÈºÜ¸ß¡£ÔÚDeepSeek´ó¹æÄ£EP²¢ÐÐall-to-allͨѶģʽÏ £¬ÍøÂçʱÑÓµÄÖ÷ÒªÓ°ÏìÒòËØÊǸºÔØÆ½ºâºÍÓµÈû¿ØÖÆ£º

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

ÈçÉÏͼËùʾ £¬ÔÚ´ó¹æÄ£EPµÄDeepSeekÍÆÀí³¡¾° £¬EPÓòµÄͨѶ¿ÉÄܺá¿ç¶à¸öLeaf £¬Á÷Á¿×ßÏòSpine £¬ÈÝÒ×±¬·¢µä·¶µÄECMP¹þÏ£²»¾ùÎÊÌâ £¬µ¼Ö½ϸ߶¯Ì¬Ê±ÑÓ¡£ÇÒDeepSeekµÄMoEÄ£×ÓÍÆÀíÒ×±¬·¢ÊµÀý¼ä¸ºÔØ·×ÆçÖºÍʵÀýÄÚר¼Ò¸ºÔØ·×ÆçÖÂÎÊÌâ £¬ÔÚÍøÂçÉÏÌåÏÖΪÁ÷Á¿ÖоÞϸÁ÷»ìÏý¡£¸ÃÕ÷Ïó¸üÈÝÒ×¼Ó¾çECMP²»¾ùµ¼ÖµĶ¯Ì¬Ê±ÑÓÎÊÌâ £¬²»¼ÑµÄ¸ºÔØÆ½ºâÕ½ÂÔ £¬ÔÚÍøÂçÉÏÈÝÒ×ÒýÈë100us+ÉõÖÁ¸ü¸ßµÄ¶¯Ì¬Ê±ÑÓ¡£ÈçÉÏÎÄÆÊÎö £¬ÕâÑùµÄ¶¯Ì¬Ê±ÑÓˮƽ¶ÔÍÌ͵ÄÓ°Ïì¿ÉÄִܵï20%+¡£ÔÚDeepSeek¹Ù·½³¡¾°ÖÐ £¬½ÓÄÉIB½»Á÷»úºÍCXÍø¿¨µÄAdaptive Routing£¨AR£©ÊÖÒÕ £¬ÓÐÓûº½âÁËECMP¸ºÔز»¾ùÎÊÌâ¡£ÔÚRoCEÇéÐÎÏ £¬¶ËÍøÐ­Í¬µÄ¸ºÔØÆ½ºâ¼Æ»®ÔÚÔÆÔÆ¿Á¿ÌµÄµÍʱÑÓÒªÇóÏ £¬ÊÇÖÁ¹ØÖ÷ÒªµÄ¡£

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

±ðµÄ £¬MoEÄ£×ӵĴó¹æÄ£×¨¼Ò²¢ÐÐͨѶʵÖÊÉÏÊÇÒ»ÖÖall-to-allģʽ £¬ÍøÂçÖÐ×ÔÈ»±£´æincastÁ÷Á¿¡£ºÏÀíµÄÓµÈû¿ØÖÆÕ½ÂÔÄܹ»×èÖ¹ÒòÁ÷Á¿½µËÙ»òPFC£¨Priority Flow Control£©´¥·¢¶ø´øÀ´µÄ¸ß¶¯Ì¬Ê±ÑÓ £¬°ü¹ÜÍøÂçʱÑÓµÄÎȹÌÐÔºÍÍÆÀíÐÔÄÜ¡£

Èý¡¢¸ßЧ¶ËÍøÔËά°ü¹Ü¸ß¿ÉÓÃÍÆÀíÓªÒµ

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

Âý¹ÊÕÏ¡¢hangÒì³£

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

Á´Â·¹ÊÕÏ

Ëæ×ÅDeepSeekÍÆÀíÒýÈë´ó¹æÄ£×¨¼Ò²¢ÐУ¨EP£© £¬ÂþÑÜÊ½ÍÆÀí¼¯ÈºÃæÁÙÓëѵÁ·¼¯ÈºÀàËÆµÄ¹ÊÕÏÌôÕ½¡£Æ¾Ö¤Meta¹ûÕæµÄÑо¿Êý¾Ý £¬ÒÔ1024¿¨¼¯ÈºÎªÀý £¬Æ½¾ùÿ7.9Сʱ»á±¬·¢Ò»´Î¹ÊÕÏ¡£ÍŽá¹ÊÕ϶ÔÍÆÀíµÄÓ°Ïì £¬¿É½«¹ÊÕÏÀàÐ͹éÄÉΪÈýÀࣺ

Âý½ÚµãÒì³££º¹ÊÕϱ¬·¢ºóÍÆÀíʹÃü²»ÖÐÖ¹ £¬µ«²¿·Ö½Úµã»ò½×¶ÎÐÔÄÜϽµ £¬µ¼ÖÂÕûÌåÍÆÀí±»ÍÏÂý £¬ÌåÏÖΪÂý½ÚµãЧӦ¡£

HangÒì³££º¹ÊÕϵ¼ÖÂÍÆÀí³¤Ê±¼ä¿¨¶ÙÓÚijһ½×¶Î £¬Ê¹ÃüÎÞ·¨¼ÌÐøÍÆ½ø £¬µ«ÕûÌåÍÆÀíÈÔδÖÐÖ¹¡£

Á´Â·¹ÊÕÏ£ºÁ´Â·ÖÐÖ¹Ö±½Óµ¼ÖÂÕû¸öÍÆÀíʵÀýÍ˳ö¡£

ÔÚÂý½ÚµãÒì³£ºÍ¶Ìʱ¼äHangÒì³£³¡¾°Ï £¬ËäÈ»ÍÆÀíʹÃüÈÔÔÚÔËÐÐ £¬µ«ÍÆÀíÐÔÄÜÏÔÖøÊÜË𠣬TTFT£¨Time To First Token£©ºÍTPOT£¨Time Per Output Token£©Ö¸±êÏÔ×Ŷñ»¯ £¬ÍÌÍÂÁ¿¿ÉÄÜϽµ50%ÒÔÉÏ¡£Òò´Ë £¬Õë¶ÔÂý¹ÊÕϺÍHangÒì³£µÄʵʱ¼à¿Ø¡¢¿ìËÙ¶¨Î»ÓëÅŲé £¬¹ØÓÚ°ü¹ÜÍÆÀíÐÔÄܾßÓÐÖ÷Òª¼ÛÖµ¡£

¶øÔÚ³¤Ê±¼äHangÒì³£»òÁ´Â·¹ÊÕϵ¼ÖÂÍÆÀíʵÀýÖ±½ÓÍ˳öµÄÇéÐÎÏ £¬ÓªÒµÓ°Ïì¸üΪÑÏÖØ¡£¹ØÓÚ´ó¹æÄ£ÊµÀý°²ÅÅÇéÐÎ £¬¿Éͨ¹ýÇëÇó¿ìËÙÇл»ÖÁÆäËû¿µ½¡ÊµÀý £¬Ëä¿ÉÄÜÎþÉü²¿·ÖÓû§ÌåÑé £¬µ«Äܰü¹ÜÓªÒµÒ»Á¬ÐÔ¡£Ïà½Ï֮Ϡ£¬ÉÙÁ¿ÊµÀý°²ÅÅ£¨Èçµ¥¸öDecodeʵÀý£©±¬·¢¹ÊÕÏʱ £¬ÍùÍùÖ±½Óµ¼ÖÂÓªÒµÖÐÖ¹ £¬ÑÏÖØÓ°ÏìÎȹÌÐÔºÍÓû§ÌåÑé¡£Òò´ËС¹æÄ£³¡¾°Ï £¬¹ÊÕϵĶ¨Î»¡¢ÌÓÉúºÍ¹æ±Ü £¬Êǰü¹ÜÓªÒµ¿ÉÓÃÐÔµÄÒªº¦ÊֶΡ£

ËÄ¡¢¸ßÐÔ¼Û±ÈÍÆÀí×éÍøÑ¹Õ¥°ÙÍòtoken±¾Ç®

1.Ë«¿ÚÍø¿¨Ë«Æ½Ãæ×éÍø£º

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

µ¥¹ìË«Æ½Ãæ×éÍø

»ùÓÚÉÏÊö¶ÔÍøÂçµÍʱÑӺ͸߿ɿ¿ÐÔµÄÐèÇó £¬½ÓÄÉÈçͼËùʾµÄµ¥¹ìË«Æ½Ãæ×éÍø¼Æ»® £¬Äܹ»×îºéÁ÷ƽ°ü¹ÜÐÔÄÜÓë¿É¿¿ÐÔ¡£Ïà±È¹Å°åCLOS¼Ü¹¹ £¬¸Ã¼Æ»®ÔÚÐÔ¼ÛÀýÈçÃæ¸ü¾ßÓÅÊÆ¡£ÏêÏ¸ÌØµãÈçÏ£º

ÓÅÊÆ£º

ÍøÂç½á¹¹¾«Á·£ºÁ÷Á¿¼¯ÖÐÓÚLeaf½»Á÷»ú £¬½µµÍ¿ç½»Á÷»úÍ¨Ñ¶ÖØÆ¯ºó £¬ÏÔÖøïÔ̭ʱÑÓ¡£

±¾Ç®Ð§Òæ¸ß£ºÖ§³ÖÍ­À»¥Áª £¬ïÔÌ­½»Á÷»úÊýÄ¿ £¬ÕûÌåÍøÂçͶÈë¸üµÍ¡£

ʱÑӵͣºÊý¾ÝÃæÁ´Â·×½öΪ2Ìø £¬×î´óÌøÊýΪ1Ìø £¬È·±£µÍʱÑÓ´«Êä¡£

Á÷¿ØÐèÇóµÍ£ºÎÞ¸ºÔØÆ½ºâÎÊÌâ £¬Á÷Á¿×ß¼òµ¥Æð¾¶ £¬¼ò»¯Á÷¿ØÉè¼Æ¡£

Ò×ÓÚÀ©Õ¹£ºÐÂÔö½ÚµãÎÞÐèÔöÌí¶þ²ãÍøÂç £¬Ö§³Ö¼¯ÈººáÏòÀ©Õ¹¡£

BondÊÊÅäÐÔÇ¿£º½ÓÄÉbondË«Æ½Ãæ×éÍøÌáÉýÍøÂç¿É¿¿ÐÔ £¬ÇÒÓÉÓÚÎÞ¶þ²ã×éÍø £¬bond¼Æ»®²»»á´øÀ´ÌØÊâ½»Á÷»ú±¾Ç®¡£

ÁÓÊÆ£º

ÎÞаÐÔÊÜÏÞ£ºPrefill»òDecodeʵÀý²»¿É¿çLeaf°²ÅÅ £¬µ¥ÊµÀý×î´ó¹æÄ£ÊÜÏÞÓÚ256¿¨¡£

¼æÈÝÐÔȱ·¦£º×éÍøÕë¶ÔÍÆÀíÁ÷Á¿ÌØÕ÷ÓÅ»¯ £¬ÄÑÒÔ¼æÈÝѵÁ·ÓëÍÆÀíÒ»Ì廯³¡¾°¡£

KV Cache´«ÊäÒÀÀµ´æ´¢Íø£ºÔÚ½ÓÄÉPDÊèÉ¢°²ÅÅʱ £¬ÈôÊDZ£´æ¿çLeafµÄPDʵÀý £¬Ôò±ØÐèÅ䱸´æ´¢ÍøÂçÒÔÖ§³ÖKV Cache´«Êä¡£

2.Shuffle¶àÆ½Ãæ×éÍø£º

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

»ùÓÚË«Íø¿ÚÍø¿¨µÄË«Æ½Ãæ×éÍø¼Æ»® £¬µ¥Pod×î´ó¹æÄ£ÊÜÏÞÓÚ256¿¨ £¬µ¼ÖÂÎÞаÐÔȱ·¦¡£ÎªÍ»ÆÆÕâһƿ¾± £¬ÔÚServerÓë½»Á÷»úÖ®¼äÒýÈëShuffle(¹â½»Ö¯ºÐ) £¬ÊµÏÖÎïÀí²ãÃæµÄ·Ö¹â¡£ÒÀÍÐ400GbpsÍø¿¨ºÍTH5оƬ½»Á÷»ú £¬×éÍø¼Æ»®Éý¼¶ÎªËÄÆ½Ãæ £¬µ¥Pod×î´ó¹æÄ£À©Õ¹ÖÁ512¿¨ £¬Öª×ã¾ø´ó´ó¶¼ÍÆÀí°²ÅÅÐèÇ󡣴˼ƻ®Ö§³Ö¸ü´ó¹æÄ£µÄEP²¢ÐкÍPDʵÀýÊýÄ¿ÔöÌí £¬ÇÒPDʵÀýÎÞÐè¿çPodµ÷Àí £¬´ó·ùÌáÉýPodÄÚ×éÍøÎÞаÐÔ £¬ÏÔÖø½µµÍ¶ÔKV Cache´æ´¢ÍøÂçµÄÒÀÀµ¡£

δÀ´ £¬Ëæ×Å800GbpsÍø¿¨ºÍTH6оƬ½»Á÷»úµÄÓ¦Óà £¬Shuffle¶à¹ì¼Æ»®¿ÉÍØÕ¹ÖÁ8¹ì¡£ÔÚ°ü¹Üµ¥GPUÏíÓÐ800Gbps´ø¿íµÄÌõ¼þÏ £¬µ¥Pod×î´ó¹æÄ£¿ÉÀ©Õ¹ÖÁ1024¿¨ £¬Öª×㳬´ó¹æÄ£ÍÆÀí·þÎñÐèÇ󡣸üƻ®ÔÚÎÞ¶þ²ã×éÍø¼Ü¹¹Ï £¬ÒÀÈ»ÌṩºÜ¸ßµÄPDÊèÉ¢°²ÅÅÎÞаÐÔ £¬PDʵÀýÎÞÐè¿çPodµ÷Àí £¬Ò²ÎÞÐèKV Cache´«ÊäרÓÃÍøÂç £¬ÊµÏÖÁË׿ԽµÄÐÔ¼Û±ÈÓëÐÔÄÜ¡£

×ܽá

DeepSeek MoEÄ£×ÓµÄÂþÑÜÊ½ÍÆÀí°²ÅÅ´øÀ´ÁËÍÆÀíÍøÂç¼Ü¹¹ºÍÐÔÄܰü¹ÜµÄÈ«ÐÂÌôÕ½¡£ÍÆÀí½×¶ÎµÄͨѶģʽºÍÁ÷Á¿ÌØÕ÷Óë¹Å°åѵÁ·±£´æÏÔÖø²î±ð £¬ÓÈÆäÊÇDecode½×¶Î¶ÔÍøÂçʱÑÓÃô¸Ð £¬ÒªÇóÍøÂç¾ß±¸µÍʱÑӺ͸ßÍÌÍÂÄÜÁ¦¡£¶ËÍøÐ­Í¬µÄ¸ºÔØÆ½ºâËã·¨ºÍÓµÈû¿ØÖÆÊÖÒÕÊǰü¹ÜÍøÂçÐÔÄܵÄÒªº¦¡£Óë´Ëͬʱ £¬ÍÆÀíÓªÒµ¸ß¿ÉÓÃÐÔÒªÇóÍêÉÆµÄ¹ÊÕÏ¼à¿Ø¡¢¿ìËÙ¶¨Î»ºÍ¹ÊÕÏÌÓÉúÕ½ÂÔ¡£Õë¶ÔÕâЩÐèÇó £¬Éè¼Æ¾«Á·¸ßЧÇҾ߱¸¸ß¿É¿¿ÐԵĵ¥¹ìË«Æ½Ãæ×éÍø¼Æ»® £¬Äܹ»ÔÚ°ü¹ÜÐÔÄܵÄͬʱ½µµÍ±¾Ç®¡£Î´À´ £¬Ëæ×ÅDeepSeek¼°ÀàËÆ´ó¹æÄ£MoEÄ£×ӵįձ鰲ÅÅ £¬ÍÆÀíÍøÂçµÄÓÅ»¯ºÍÁ¢Ò콫³ÉΪ½¹µã¾ºÕùÁ¦¡£

Ïà¹Ø±êÇ©£º

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾ 97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

µãÔÞ

¸ü¶àÊÖÒÕ²©ÎÄ

ÈκÎÐèÒª £¬ÇëÁªÏµ97¹ú¼Ê

97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾

·µ»Ø¶¥²¿

ÊÕÆð
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾ ÎĵµAIÖúÊÖ
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾ ÎĵµÆÀ¼Û
¸Ã×ÊÁÏÊÇ·ñ½â¾öÁËÄúµÄÎÊÌ⣿
Äú¶ÔÄ¿½ñÒ³ÃæµÄÖª×ã¶ÈÔõÑù£¿
²»Õ¦µÎ
ºÜÊǺÃ
ÄúÖª×ãµÄÔµ¹ÊÔ­ÓÉÊÇ£¨¶àÑ¡£©£¿
Äú²»Öª×ãµÄÔµ¹ÊÔ­ÓÉÊÇ£¨¶àÑ¡£©£¿
ÄúÊÇ·ñÉÐÓÐÆäËûÎÊÌâ»ò½¨Ò飿
ΪÁË¿ìËÙ½â¾ö²¢»Ø¸´ÄúµÄÎÊÌâ £¬Äú¿ÉÒÔÁôÏÂÁªÏµ·½·¨
ÓÊÏä
ÊÖ»úºÅ
ллÄúµÄ·´À¡£¡
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
97¹ú¼Ê¡¤(ÖйúÇø)¼¯ÍŹٷ½ÍøÕ¾
ÇëÑ¡Ôñ·þÎñÏîÄ¿
¹Ø±Õ×Éѯҳ
ÊÛǰ×Éѯ ÊÛǰ×Éѯ
ÊÛǰ×Éѯ
ÊÛºó·þÎñ ÊÛºó·þÎñ
ÊÛºó·þÎñ
Òâ¼û·´Ïì Òâ¼û·´Ïì
Òâ¼û·´Ïì
¸ü¶àÁªÏµ·½·¨
ÍøÕ¾µØÍ¼