Ukungena ezweni lobuhlakani bokwenziwa bendawo kungase kubonakale njengomsebenzi oyinkimbinkimbi ekuqaleni, kodwa iqiniso liwukuthi sisesikhathini segolide. Awusadingi isikhungo sedatha esingaphansi komhlaba ukuze ukwenze. incazelo yemodeliNamuhla, ngethimba elihlelwe kahle, ungaba nomsizi wakho wangasese ongaxhomekile efwini noma okubhaliselwe kwanyanga zonke.
Uma sikhuluma ngokusebenzisa amamodeli endaweni, empeleni sibhekisela esigabeni ukucabangelaYilapho imodeli eqeqeshwe ngaphambilini icubungula idatha entsha ukuze inikeze impendulo. Ngokungafani nokuqeqeshwa, okuwumsebenzi omkhulu odinga izinkulungwane zama-GPU, ukuqagela kulula kakhulu ukulawuleka, futhi yilapho ama-chip e-Apple enza khona umthelela omkhulu ngenxa yawo ukuphathwa kwamandla ngendlela efanele kanye nokwakhiwa kwayo okusha.
Imfihlo yempumelelo: Inkumbulo Ehlanganisiwe
Uma kukhona into eyodwa eyenza ama-Mac avelele, yi- Ukwakhiwa Kwememori Okuhlanganisiwe (UMA)Ku-PC yendabuko, kufanele uthuthe idatha kusuka ku-RAM yesistimu uye ku-VRAM yekhadi lehluzo, futhi yilapho isikhathi esiyigugu silahleka khona. Ku-M4, M3 Ultra, noma ama-chip afanayo, i-CPU ne-GPU zidonsa kusuka echibini lememori elifanayo, eli kunciphisa ukubambezeleka kakhulu lapho kusingathwa ama-tensioner.
Lokhu kubalulekile uma sifuna ukulayisha ama-LLM aphakathi noma amakhulu (Amamodeli Olimi Olukhulu). Isibonelo, i-Mac Studio ene-RAM eningi ingaphatha amamodeli angadinga okuningi kakhulu kwi-PC. amakhadi amaningana e-NVIDIA A6000Lokhu kusho ukonga okukhulu ezindleleni, futhi ngaphezu kwakho konke, ebhilini likagesi, njengoba ukusetshenziswa kuyingxenye encane yalokho umbhoshongo wokugembula ongakusebenzisa.
Ukulinganisa kanye namafomethi: Indlela yokwenza imodeli ifaneleke
Ukuze sivimbele imodeli enamapharamitha ayizigidigidi ezingama-70 ukuthi ingashayi umshini wakho, siphendukela ku- quantization. Básicamente, consiste en reducir la precisión de los números (pasar de FP32 a INT8 o incluso 4 bits), lo que encoge el modelo y acelera la respuesta sin que el modelo se vuelva «tonto» o pierda coherencia.
- I-GGUF: Yifomethi ekhethwayo kulabo abasebenzisa i-CPU noma inhlanganisela ye-CPU ne-GPU. Yifayela elilodwa eliqukethe konke okudingekayo futhi liyindinganiso ye shayela.cpp.
- I-GPTQ: Yakhelwe ngqo ukuhamba kuma-GPU, okuthuthukisa isivinini sokucubungula.
- Izinto zokuvikela: Inketho ephephe kakhulu kunamafayela e-.bin avamile, njengoba ivimbela ukwenziwa kwekhodi enonya lapho kulayishwa imodeli.
Amathuluzi abalulekile okusetha indawo yakho
Uma ungafuni ukuhlushwa yi-terminal kusukela ngomzuzu wokuqala, kunezinketho ezilula kakhulu. I-LM Studio es probablemente la herramienta más amigable: es un «todo en uno» con interfaz gráfica que te permite sebenzisa amamodeli e-AI endaweni bese uziqalisa ngokuchofoza okukodwa. Ungakwazi ngisho nokusetha i- iseva ye-API yendawo Iyahambisana ne-OpenAI ukuhlanganisa i-AI kuzinhlelo zakho zokusebenza.
Ngakolunye uhlangothi esinakho U-Ollamaokuyiyona nto eyigugu kulabo abathanda into elula noma esuselwe kumyalo. Iwumthombo ovulekile futhi ihlangana kahle ne Vula i-WebUIokukuvumela ukuthi ube nesixhumi esibonakalayo esifana ne-ChatGPT kodwa esisebenza ngokuphelele kwihadiwe yakho nge- Umhlahlandlela osheshayo wokusebenzisa ama-LLM endawo nge-OllamaKulabo abafuna ukusebenza okuphezulu ku-macOS, uhlaka I-Apple MLX Kuyindlela elungiselelwe kahle yokuthola okuningi nge-silicon yomkhiqizo.
Inkinga Yehadiwe: Ukuphatheka Kalula Noma Amandla Aluhlaza?
Abacwaningi abaningi nabathuthukisi bahlukene phakathi kwezindlela ezimbili. Inketho yokuqala ukufaka konke ndawonye IMacBook Pro M4 Max enememori eningi (njenge-128GB). Inzuzo ukuthi ukuphindaphinda okusheshayoUngaba sekhefini noma endizeni uhlola ipayipi le-RAG (Recovery Augmented Generation) ngaphandle kokuthembela ekuxhumekeni okukude.
Okunye okuhlukile kuyinhlanganisela ye IMacStudio kanye ne-laptop elula. I-Studio iyisilwane esishisayo; ungayishiya isebenza isikhathi eside amahora amaningi ngaphandle kokuthi ifeni izwakale njengenjini yejethi isuka. Kodwa-ke, lokhu kwethula ukungqubuzana kokufinyelela kudeUkuvumelanisa izindawo kanye nedatha phakathi kwemishini emibili ehlukene kungaphazamisa ukugeleza kokudala.
Amacala okusetshenziswa kwangempela kanye nemikhawulo
Nge-Mac Mini M4 noma i-MacBook Pro ehlonyiswe kahle, ungakha izinhlelo ze- I-RAG enezizindalwazi ze-vector njenge-ChromaDB noma i-Qdrant, dala abasizi bekhodi yendawo nge-Aider noma Bhala amavidiyo ngokuzenzakalelayo usebenzisa i-AI yendawo yokukhipha izinto. Ilungele ukwenza iphrothotayipi nokuhlola ukuziphatha kwamamodeli alinganisiwe anamapharamitha aphakathi kuka-7B no-70B.
Nokho, akufanele sizame ukwenza izimangaliso. Ukuqeqeshwa okujulile kwemodeli noma inkimbinkimbi yokulungisa kahle La maqembu awawona amaphuzu aqinile. Uma umsebenzi wakho uncike ngokuphelele ku Izindawo zemvelo ze-CUDA Nge-NVIDIA, uzohlangana nezithiyo ezithile endleleni, njengoba i-Metal (i-Apple's API) ingeyona indawo ecacile, yize uma sibheka ukuthi igebe linciphe kakhulu.
Umbhali oshisekayo ngomhlaba wamabhayithi nobuchwepheshe ngokujwayelekile. Ngiyathanda ukwabelana ngolwazi lwami ngokubhala, futhi yilokho engizokwenza kule bhulogi, ngikubonise zonke izinto ezithakazelisayo kakhulu ngamagajethi, isofthiwe, ihadiwe, izitayela zobuchwepheshe, nokuningi. Inhloso yami ukukusiza ukuthi uzulazule emhlabeni wedijithali ngendlela elula nejabulisayo.