- Ukusetshenziswa kwamaModeli Olimi Oluncane (i-SLM) kanye nokulinganisa ukuze kusetshenziswe i-AI kuhadiwe yendawo ngaphandle kokuthembela efwini.
- Ukusebenzisa ukwakheka kwe-RAG kanye nezizindalwazi ze-vector ukuthola izimpendulo ezinembile ezisekelwe kumadokhumenti ayimfihlo.
- Izidingo zehadiwe zigxile kumemori ehlanganisiwe ye-NVIDIA GPU VRAM kanye ne-Apple Silicon.
- Izinzuzo zobumfihlo kanye nokuzimela kwedatha ngokufaka inkomba ye-semantic kumaseva e-NAS ebhizinisi.
Cabanga nje ukuthi unamandla obuchopho bedijithali obucubungula yonke imininingwane yakho yangasese ngaphandle kokuphuma ehhovisi lakho. Kuze kube muva nje, ukusetha uhlelo lobuhlakani bokwenziwa ekhaya kwakuyinto ebuhlungu efanele unjiniyela we-NASA, elwa nokuthembela kumakhodi kanye nehadiwe eshisa kakhulu kune-toaster. Kodwa-ke, isimo sishintshe kakhulu, futhi namuhla kungenzeka ngokuphelele ukufaka amamodeli amancane olimi emishinini yendawo, ukuguqula isitoreji samafayela esilula sibe umsizi ohlakaniphile.
Uguquko lwangempela luza nokuhlanganiswa kwesitoreji esinamathiselwe kunethiwekhi (i-NAS) kanye nokufunda komshini onqenqemeni. Asisakhulumi ngamadivayisi alula, angachazwanga agcina kuphela o-zero kanye no-one, kodwa ngamaseva ahlakaniphile anama-NPU akwazi ukuqonda okusesithombeni noma ukufingqa inkontileka esemthethweni ngemizuzwana. Leli khono lokucubungula idatha ngaphandle kokuthembela efwini akuyona nje indaba yokulula, kodwa liyisivikelo esiqinile sobumfihlo banoma yiliphi ibhizinisi noma umsebenzisi ngamunye, okwenza ngcono isitoreji sendawo ngaphezu kwe-cloud computing.
Ukuvela kwe-NAS: kusuka ku-hard drive kuya ebuchosheni bedijithali
Isitoreji sendabuko besilokhu siyinkinga enkulu iminyaka eminingi. Amabhizinisi amancane naphakathi nendawo (ama-SME) avame ukuqongelela ama-terabyte ama-PDF, izithombe, namavidiyo agcina efihliwe kumafolda anamagama angenangqondo njenge-"final_v2_this_is_it." Inkinga ukuthi i-NAS evamile ayikwazi ukubona ukuthi isithombe singesesikrini sokumaketha; ibona ifayela kuphela. Yilapho i -AI NAS ingena khona , isebenzisa amaprosesa athuthukile njenge-AMD Ryzen enamayunithi okucubungula ama-neural (ama-NPU) ukuze ikhombise okuqukethwe ngokwesisho.
Ngenxa yalokhu, singakwazi ukusebenzisa usesho lwe-semantic . Esikhundleni sokusesha igama lefayela eliqondile, ubuza iseva ukuthi, "Ngitholele izithombe zobuciko emvuleni," bese i-AI, esevele ihlaziye izithombe isebenzisa ama-Visual Language Models (VLM), ibuyisa imiphumela eqondile. Lolu hlelo luvumela ihadiwe, njengochungechunge lwe-Minisforum N5, ukuthi yenze ukuqashelwa kwezinto ngesikhathi sangempela kanye ne-OCR , isindise amahora omsebenzi wezandla wamaqembu okudala noma okuphatha.
Amamodeli Olimi Oluncane (i-SLM) vs. Amamodeli Olimi Olukhulu (i-LLM)
Ukuze konke lokhu kusebenze kuseva yendawo ngaphandle kokuphahlazeka kwesistimu, kusetshenziswa amaModeli Olimi Oluncane (ama-SLM) . Ngenkathi ama-LLM (njenge-GPT-4) enezigidigidi zamapharamitha futhi edinga amapulazi eseva, ama-SLM ngokuvamile anemingcele engaphansi kwezigidigidi eziyi-10. Amamodeli afana ne- Microsoft's Phi-3, i-Mistral 7B, noma i-Gemma afaneleka kakhulu ekusetshenzisweni kwendawo ngoba ashesha kakhulu, asebenzisa i-RAM encane, futhi awathambekele ekulungiseni ngokweqile.
Isihluthulelo sokwenza la mamodeli asebenzise ihadiwe encane yi -quantization . Le ndlela ihilela ukunciphisa ukunemba kwezisindo zemodeli (isibonelo, kusukela ku-FP32 kuya ku-INT8 noma ku-INT4), okunciphisa kakhulu ukusetshenziswa kwememori ngaphandle kokwenza imodeli ingabi hlakaniphile. Lokhu kuvumela imodeli enamandla ukuthi ilingane ne-VRAM yekhadi lehluzo labathengi noma imemori ehlanganisiwe ye-Mac, okusheshisa ukuqagela nokwenza izimpendulo zicishe zisheshe.
Izakhiwo ezithuthukisiwe: i-RAG kanye nokusetshenziswa kobuchwepheshe
Ukuze kuvinjelwe i-AI ekuqambeni izinto (imibono edumile yokubona izinto ezingekho emthethweni), kusetshenziswa inqubo ebizwa ngokuthi i-Recall Augmented Generation (RAG ). Esikhundleni sokuthembela kuphela kulokho imodeli ekufundile ngesikhathi sokuqeqeshwa, uhlelo lusika amadokhumenti akho endawo, luwaguqule abe ama-vector ezinombolo (ukushumeka), bese luwagcina kudathabheyisi ye-vector efana ne-FAISS . Uma ubuza umbuzo, uhlelo lusesha amafayela akho ukuze luthole ingxenye yombhalo efanele kakhulu bese luyidlulisela kumodeli ukuze ikhiqize impendulo esekelwe ebufakazini bangempela bendawo.
Uma ukulungele ukufaka ikhodi, ungasetha le ndawo usebenzisa i-FastAPI ye-interface kanye ne-LangChain ukuphatha izixwayiso. Ukuhamba komsebenzi okuvamile kungabandakanya ukulayisha imodeli elinganiselwe usebenzisa umtapo wezincwadi we-Hugging Face's Transformers, ukuxhuma kusizindalwazi se-vector, nokuveza konke nge-REST API. Kulabo abakhetha ukungabhali ikhodi, kunezindlela ezifana nokusebenzisa ama-LLM endawo nge-Ollama noma i-LM Studio ezilawula ukulanda nokusebenza kwamamodeli ngokuchofoza okukodwa, ngisho nokukuvumela ukuthi ushintshe phakathi kwamamodeli endawo kanye nezinsizakalo zamafu kuye ngokuthi idatha yakho ibucayi kangakanani.
Ihadiwe enconyiwe ye-AI yendawo
Ihadiwe yimbangela eyinhloko. Kuhlelo lwe-PC, i-GPU VRAM iyinkosi; i-RTX 4090 enama-24GB iyindinganiso yegolide yamamodeli asebenza ngokunethezeka anamapharamitha afinyelela ku-30B. Ngakolunye uhlangothi, ama-chip e-Apple Silicon (M2/M3/M4) asebenza kahle ngokumangazayo ngenxa yememori yawo ehlanganisiwe, enza kube lula ukuqagela kwe-AI yendawo ku-macOS , okuvumela i-GPU ukufinyelela yonke i-RAM yesistimu, okwenza kube lula ukusebenzisa amamodeli amakhulu kunakwi-PC evamile.
- Ibanga lokokufaka: Izinhlelo ezine-RAM engu-16 GB kanye nama-GPU aphansi amamodeli anamapharamitha angu-3B kuya ku-7B.
- Ububanzi obuphakathi: Amadivayisi e-NAS noma ama-Mac anamandla ane-RAM engu-32-64 GB yamamodeli angu-13B kuya ku-20B anokulinganiselwa.
- Uhla lochwepheshe: Izindawo zokusebenza ze-Multi-GPU (A6000 noma 4090) zamamodeli angu-70B noma ngaphezulu.
Imikhiqizo efana ne-QNAP isivele ihlanganisa izici ezifana ne -Qsirch AI Mode , ehlanganisa i-VLM ne-LLM ukuze ifinyeze amadokhumenti, ihumushe imibhalo futhi ithole izikhathi eziqondile kumavidiyo noma kuma-audio kusetshenziswa ama-transcription azenzakalelayo (i-ASR), konke lokhu ngenkathi ihlonipha izimvume zomsebenzisi futhi ngaphandle kwedatha ephuma kunethiwekhi yendawo.
Amasu okusebenzisa kanye nokuphepha
Ukuqalisa uhlelo lwe-AI olusezindaweni enkampanini kungaphezu nje kokufaka isofthiwe. Kubalulekile ukulandela umthetho wokusekela ngokulondoloza we-3-2-1 (amakhophi amathathu, amabili kwimidiya yokugcina, elilodwa ngaphandle kwendawo) ukuze ugweme ukulahlekelwa yinkomba ye-AI uma ihadiwe yehluleka, njalo uhlola isu elihle kakhulu lokusekela ngokulondoloza phakathi kwendawo yokugcina nokugcina amafu . Ngaphezu kwalokho, kuyalulekwa ukukhomba idatha ngamaqoqo ubusuku bonke ukuze ugweme ukulayisha ngokweqile i-bandwidth yehhovisi ngenkathi abasebenzi besebenza.
Ukuthunyelwa ezindaweni eziyinkimbinkimbi kakhulu kungasekelwa yi -Kubernetes (AKS) kanye nabaqhubi abafana ne-KAITO, abazenzakalela ukuphathwa kwama-node e-GPU kanye nokukala imodeli. Lokhu kuqinisekisa ingqalasizinda eqinile futhi kuvimbela i-AI ukuthi ingaphazamisi lapho abasebenzisi abaningi beqhuba imibuzo ngesikhathi esisodwa. Ubukhosi bedatha buyimpahla enkulu lapha: ngokuqeda ukuthembela ekubhaliseni kwanyanga zonke nokuvimbela abahlinzeki bamafu ekuqeqesheni amamodeli abo ngedatha yakho , uthola ukulawula okugcwele impahla yakho yobuhlakani.
Ukuhlangana kwehadiwe ekhethekile, amamodeli amancane alungiselelwe ngokusebenzisa i-quantization, kanye nezakhiwo zokubuyisa idatha zasendaweni manje kuvumela ukudalwa kwezinhlelo zokusebenza lapho ubumfihlo bubaluleke kakhulu. Ngokuhlanganisa la mathuluzi ku-NAS noma i-workstation enamandla, ukuphathwa kolwazi kuguqulwa kube inqubo esebenzayo neqondakalayo, kususa ukungqubuzana kokusesha ngesandla nokuqinisekisa ukuthi ulwazi lwenkampani luhlala lutholakala futhi luvikelwe.
Umbhali oshisekayo ngomhlaba wamabhayithi nobuchwepheshe ngokujwayelekile. Ngiyathanda ukwabelana ngolwazi lwami ngokubhala, futhi yilokho engizokwenza kule bhulogi, ngikubonise zonke izinto ezithakazelisayo kakhulu ngamagajethi, isofthiwe, ihadiwe, izitayela zobuchwepheshe, nokuningi. Inhloso yami ukukusiza ukuthi uzulazule emhlabeni wedijithali ngendlela elula nejabulisayo.