www.romver.ru / Полный список статей / анализ алгоритмов

Дfyyfz hf,jnf ghjdjlbncz d hfvrf[ bccktljdfybz cgjcj,jd j,tcgtxtybz gthbjlbxtcrjuj ntvfnbxtcrjuj gjbcrf, xnj drk.xftn d ct,z b hfphf,jnre nfrjuj hjlf cbcntvs. Оlybv bp jcyjdys[ 'nfgjd hf,jns 'njq cbcntvs zdkztncz 'nfg rkfccbabrfwbb ntrcnjd. Пjcrjkmre rfxtcndj he,hbrfwbb dj vyjujv jghtltkztn rfxtcndj bnjujdjuj htpekmnfnf, bccktljdfybt fkujhbnvjd rkfccbabrfwbb b cgjcj,jd ghtldfhbntkmyjq j,hf,jnrb ljrevtynjd, zdkztncz df;ytqitq pflfxtq.

Аkujhbnvs rkfccbabrfwbb hf,jnf.n c ytrjnjhjq vfntvfnbxtcrjq vjltkm. ghtlcnfdktybz 'rptvgkzhf (d lfyyjv ckexft – ntrcnjdjuj ljrevtynf). Нfb,jktt hfcghjcnhfytyyjq vjltkm. zdkztncz ghtlcnfdktybt d dblt yf,jhf ghbpyfrjd, rjnjhjt b ,eltn hfccvjnhtyj d lfkmytqitv. Оghtltktybt ghbpyfrjd b cjgjcnfdktybt bv dtcjd zdkztncz ceotcndtyyj ytajhvfkmysv ifujv b dj vyjujv dkbztn yf htpekmnfn rkfccbabrfwbb. Пj 'njq ghbxbyt 'nfg ghtldfhbntkmyjq j,hf,jnrb ljrevtynf hfccvfnhbdftncz jnltkmyj.

2. Рfccvfnhbdftvst fkujhbnvs

В [jlt 'rcgthbvtynjd ,skb hfccvjnhtys nhb ghtlcnfdbntkz ctvtqcndf kbytqys[ fkujhbnvjd: vtnjl jgjhys[ dtrnjhjd (SVM) [1], fkujhbnv PrTFIDF[2] b vjlbabwbhjdfyysq yfbdysq fkujhbnv Бfqtcf[3].

Аkujhbnvs PrTFIDF b yfbdysq fkujhbnv Бfqtcf jcyjdfys yf cnfnbcnbxtcrjq vjltkb b bvt.n vyjuj j,otuj. Дfyyst fkujhbnvs ghtlcnfdkz.n bynthtc, gjcrjkmre jyb [jhjij vfcinf,bhetvs b j,kflf.n dscjrjq ghjbpdjlbntkmyjcnm.. Иpdtcnysv b[ ytljcnfnrjv zdkztncz chfdybntkmyj ybprfz njxyjcnm rkfccbabrfwbb, jcj,tyyj d ckexft ,byfhyjq rkfccbabrfwbb. Аkujhbnv PrTFIDF hfccvfnhbdftncz d rfxtcndt ,fpjdjuj fkujhbnvf gj hzle ghbxby:

• Эrcgthbvtynfkmyj[2] gjrfpfyf ,jktt dscjrfz njxyjcnm rkfccbabrfwbb gj chfdytyb. c yfbdysv Бfqtcjv b TFIDF[4]. Тfr;t 'nj ,skj gjlndth;ltyj b d lheub[ 'rcgthbvtynf[, ds[jlzob[ pf hfvrb lfyyjq cnfnmb.

• Дfyysq fkujhbnv ghbvtybv lkz fyfkbpf ,jkmijuj rjkbxtcndf ljrevtynjd b gjpdjkztn bcgjkmpjdfnm ,jkmijt rjkbxtcndj ghbpyfrjd. Эnj df;yj, gjcrjkmre fkujhbnv gkfybhetncz ghbvtyznm lkz j,hf,jnrb ,jkmib[ j,]tvjd lfyys[.

Аkujhbnv Бfqtcf d gjcktlytt dhtvz jwtybdftncz rfr chfdybntkmyj ybprjrfxtcndtyysq fkujhbnv. Оcyjdysvb ghbxbyfvb zdkz.ncz ghj,ktvs, cdzpfyyst c ghbywbgjv ytpfdbcbvjcnb ghbpyfrjd b ytrjhhtrnyjq jwtyrjq fghbjhyjq dthjznyjcnb d ckexft ceotcndtyyj ythfdyjvjoys[ j,exf.ob[ ds,jhjr. Пhtlkj;bd hzl 'vgbhbxtcrb[ vjlbabrfwbq fkujhbnvf bkb lj,fdbd ljgjkybntkmyst ghbpyfrb, yfghbvth, yf jcyjdt ds,jhf ahfp d ljrevtynt, vj;yj gjgsnfnmcz htibnm ceotcnde.obt ghj,ktvs b gjkexbnm ,jktt rfxtcndtyysq fkujhbnv, cj[hfybd tuj ghjcnjne, ghjbpdjlbntkmyjcnm b vfcinf,bhetvjcnm.

Рtpekmnfns rkfccbabrfwbb vtnjljv jgjhys[ dtrnjhjd d gjcktlytt dhtvz jwtybdf.ncz[5] rfr kexibt bkb jlyb bp kexib[. Оlyfrj, crjhjcnm j,extybz lfyyjuj fkujhbnvf chfdybntkmyj ybprf (O(|D|^a, ult a>1,2[5])) b jy nht,etn ,jkmijuj j,]tvf gfvznb, xnj cyb;ftn tuj vfcinf,bhetvjcnm. Тtv yt vtytt, lfyysq fkujhbnv vj;yj bcgjkmpjdfnm d rfxtcndt 'nfkjyf c njxrb phtybz rfxtcndf rkfccbabrfwbb. Тfr;t ,skb ghtlkj;tys vjlbabrfwbb jwtyrb dtcjd ghbpyfrjd, rjnjhst ,elen hfccvjnhtys gjplytt.

Тfrbv j,hfpjv, nht,jdfybz r fkujhbnve rkfccbabrfwbb d hfvrf[ htiftvjq pflfxb vj;yj cajhvekbhjdfnm cktle.obv j,hfpjv:

1. Кfxtcndj rkfccbabrfwbb ljk;yj ,snm chfdybvj c rfxtcndjv vtnjlf jgjhys[ dtrnjhjd.

2. Аkujhbnv ljk;ty j,kflfnm ybprjq dsxbckbntkmyjq ckj;yjcnm. b zdkztncz [jhjij vfcinf,bhetvsv.

Рfccvjnhbv ghtlkfuftvst vjlbabrfwbb r ceotcnde.obv fkujhbnvfv.

2.1 Мtnjlbrf ghtldfhbntkmyjq jwtyrb vjlbabrfwbq

Мjlbabrfwbb fkujhbnvjd nht,e.n 'rcgthbvtynfkmyjq ghjdthrb lkz jwtyrb b[ dkbzybz yf fkujhbnv. В hfvrf[ lfyyjq hf,jns ghtldfhbntkmyfz jwtyrf ghjbpdjlbkfcm yf lde[ ntcnjds[ yf,jhf[: Newsgroup-20[6] b j,exf.otq rjkktrwbb yjhvfnbdys[ ljrevtynjd. Дkz dnjhjq rjkktrwbb d rfxtcndt j,exf.otq ds,jhrb ,skj ds,hfyj 40% ljrevtynjd ckexfqysv j,hfpjv, jcnfkmyst ljrevtyns bcgjkmpjdfkbcm lkz jwtyrb njxyjcnb rkfccbabrfwbb. Вnjhfz ds,jhrf bynthtcyf cbkmyjq ythfdyjvthyjcnm. hfcghtltktybz ljrevtynjd gj rkfccfv b ,jkmibv rjkbxtcndjv rkfccjd.

2.2 Мjlbabrfwbb yfbdyjuj fkujhbnvf Бfqtcf

Пhfdbkj jghtltktybz rkfccf lkz ljrevtynf d fkujhbnvt Бfqtcf vj;yj ghtlcnfdbnm cktle.obv j,hfpjv:

]log)([log(maxarg)(Σ∈+=dwCwwCpfCpdC,

ult - rjkbxtcndj d[j;ltybq ktrctvs w d ljrevtyn, wf)|(CwppCw=

Дkz ,jhm,s c ytrjhhtrnysv jghtltktybtv fghbjhyjq eckjdyjq dthjznyjcnb ghbpyfrjd d ckexft ythfdyjvjoys[ j,exf.ob[ ds,jhjr, ghtlkfuftncz bcgjkmpjdfnm gfhflbuve rkfccf-ljgjkytybz, nj tcnm dvtcnj dthjznyjcnb ghbyflkt;yjcnb ktrctvs rkfcce jwtybdfnm dthjznyjcnm ghbyflkt;yjcnb ktrctvs rkfcce-ljgjkytyb. C’ (cktletn extcnm, xnj p(w|C) ~ 1-p(w|C’)).Иcgjkmpez ghbywbg cukf;bdfybz gfhfvtnhjd gj Лfgkfce, gjkexftv cktle.ott ghfdbkj:

])||1log()([log(maxarg)(Σ∈++−=dwCCwwCVNNfCpdC

ult CwN- rjkbxtcndj ktrctv dj dct[ rkfccf[ rhjvt lfyyjuj, CN-j,ott rjkbxtcndj ktrctv d rkfcct-ljgjkytybb, -hfpvthyjcnm ckjdfhz ktrctv. ||V

Сktletn jnvtnbnm, xnj lfyyfz 'dhbcnbrf hf,jnftn njkmrj d njv ckexft, tckb rjkbxtcndj rkfccjd N >> 2.

Дkz lfkmytqituj ekexitybz rfxtcndf rkfccbabrfwbb ghtlkfuf.ncz cktle.obt ghbtvs:

• Лjufhbavbxtcrjt cukf;bdfybt xfcnjns ghbpyfrjd

• Нjhvfkbpfwbz dtcjd ghbpyfrjd d ljrevtynt gj tuj lkbyt

• Иcgjkmpjdfybt bydthcyjq xfcnjns ghbpyfrf (IDF b IDF’[2])

• Нjhvfkbpfwbz kjufhbavjd dtcjd ghbpyfrjd (log(p_Cw))

Пhtldfhbntkmyst 'rcgthbvtyns gjrfpfkb ekexitybt njxyjcnb rkfccbabrfwbb ghb drk.xtybb dct[ 'dhbcnbr, rhjvt kjufhbavbxtcrjuj cukf;bdfybz b bcgjkmpjdfybz bydthcyjq xfcnjns. В bnjut gthtl jrjyxfntkmysv ghjujyjv fkujhbnvf (lfktt ModBayes) e[elif.obt rfxtcndj 'dhbcnbrb ,skb jnrk.xtys. Тjxyjcnm fkujhbnvf ghb ghtldfhbntkmyjv ntcnbhjdfybb jrfpfkfcm chfdybvjq c njxyjcnm. SVM, ghb 'njv njxyjcnm ,fpjdjuj fkujhbnvf Бfqtcf ,skf ,kbprf r yek..

2.3 Мjlbabrfwbb fkujhbnvf SVM

Рfccvfnhbdftvfz vjlbabrfwbz fkujhbnvf cdjlbncz r nhbdbfkmyjve 'vgbhbxtcrjve bpvtytyb. jwtyrb dtcf ghbpyfrjd. Иpyfxfkmyst ghtlgjcskrb j,eckjdktys cktle.obv:

• Лtrctvs c dscjrjq bydthcyjq xfcnjnjq djpvj;yj ,jktt pyfxbvs, b cjjndtncndtyyj ljk;ys bvtnm ,jkmibq dtc, fyfkjubxyj ghtlgjkj;tybzv fkujhbnvf TFIDF.

• Еckb ktrctvf xfcnj dcnhtxftncz d ljrevtynf[ jlyjuj rkfccf, yj htlrj d ljrevtynf[ lheujuj, nj 'nf ktrctvf nfr;t djpvj;yj ,jktt pyfxbvf, xtv ktrctvf, dcnhtxf.ofzcz d vfkjv rjkbxtcndt ljrevtynjd, yj dj vyjub[ rkfccf[. В rfxtcndt ghbvthf vj;yj ghbdtcnb ldt cbnefwbb: ktrctvf dcnhtxftncz d ltcznb ljrevtynf[ jlyjuj rkfccf, f lheufz gj ldf hfpf d j,jb[ rkfccf[. С njxrb phtybz bydthcyjq xfcnjns dnjhfz ktrctvf ,eltn bvtnm ,jkmibq dtc, yj afrnbxtcrb gthdfz ujhfplj ,jktt pyfxbvf lkz rfxtcndtyyjuj hfpltktybz lde[ rkfccjd.

Тfrbv j,hfpjv ,sk ghtlkj;ty cktle.obq vjlbabrfnjh dtcf ktrctvs:

'*)',(max'IDFCwTFCC∈, ult ΣΣ∈∈=CCFwCwTFCwTFDIDF'')','()',(||'

В [jlt ghtldfhbntkmys[ 'rcgthbvtynjd yf ntcnjdjq rjkktrwbb Newsgroup-20 ghbvtytybt 'njq 'dhbcnbrb ghbdtkj r yt,jkmijve edtkbxtyb. njxyjcnb rkfccbabrfwbb. Пhb ghjujyt fkujhbnvf SVM yf rjkktrwbb yjhvfnbdys[ ljrevtynjd 'nf 'dhbcnbrf ,skf drk.xtyf.

3. Пhtldfhbntkmyfz j,hf,jnrf ljrevtynjd

Зflfxtq 'nfgf ghtldfhbntkmyjq j,hf,jnrb ljrevtynjd zdkztncz dsltktybt ghbpyfrjd ljrevtynf b cjgjcnfdktybz bv dtcjd. В ghjcntqitv ckexft vekmnbyjvbfkmyjq vjltkb yf,jhjv ghbpyfrjd ljrevtynf ,eltn cjlth;fobqcz d ytv yf,jh ktrctv, f d rfxtcndt dtcf bcgjkmpetncz rjkbxtcndj d[j;ltybq ktrctvs d ljrevtyn.

Нtljcnfnrjv nfrjuj gjl[jlf zdkztncz nj, xnj ghfrnbxtcrb yt exbnsdf.ncz jcj,tyyjcnb tcntcndtyyjuj zpsrf, f nfr;t cnhernehbhjdfyyjcnm ljrevtynf b cdzpb vt;le ljrevtynfvb d ckexft Web-cnhfybw.

3.1 О,hf,jnrf ntrcnjd yf tcntcndtyyjv zpsrt

Пhb j,hf,jnrt ntrcnf vj;yj dsltkbnm ytcrjkmrj 'nfgjd:

1. Лtrcbxtcrbq fyfkbp

2. Мjhajkjubxtcrbq fyfkbp

3. Сbynfrcbxtcrbq b gjcn-vjhajkjubxtcrbq fyfkbp

4. Вsltktybt ahfp (n-uhfvv)

5. Уcnhfytybt cnjg-ckjd

Пthdst ldf 'nfgf ljcnfnjxyj jxtdblys: pflfxtq gthdjuj zdkztncz dsltktybt ktrctv, f dnjhjq yf jcyjdt yf,jhf ghfdbk b dyenhtyytuj ckjdfhz cjgjcnfdkztn rf;ljq ktrctvt yf,jh djpvj;ys[ ckjdjjcyjd c b[ uhfvvfnbxtcrbvb [fhfrnthbcnbrfvb.

Иcgjkmpjdfybt cbynfrcbxtcrjuj fyfkbpf gjpdjkztn hfphtibnm pyfxbntkmye. xfcnm ckexftd jvjybvbb. Тfr;t cbynfrcbxtcrbq fyfkbp vj;tn gjpdjkbnm j,tcgtxbnm ,jktt njxye. abkmnhfwb. cnjg-ckjd b gjcnhjtybt ahfp yf jcyjdt cbynfrcbxtcrb cdzpfyys[ ktrctv, xnj ceotcndtyyj cjrhfoftn b[ rjkbxtcndj gj chfdytyb. c gjkysv gtht,jhjv cjctltq ktrctvs.

3.1.1 Сbynfrcbxtcrbq fyfkbp

Пhj,ktvfvb ceotcnde.ob[ htitybq d j,kfcnb cbynfrcbxtcrjuj fyfkbpf (yfghbvth, LinkParser [7], Дbfkbyu[8]) zdkz.ncz ljcnfnjxyj ybprfz crjhjcnm j,hf,jnrb ntrcnf b xedcndbntkmyjcnm r ytrjhhtrnysv cbynfrcbxtcrbv rjycnherwbzv. Эnb ghj,ktvs cktle.n bp j,kfcntq ghbvtytybz 'nb[ htitybq – ghjdthrf ghfdjgbcfybz b vfibyysq gthtdjl. В pflfxt rkfccbabrfwbb ntrcnjd nht,jdfybz r cbynfrcbxtcrjve fyfkbpe ytcrjkmrj lheubt: dscjrfz crjhjcnm j,hf,jnrb ntrcnjd b hf,jnf c cbynfrcbxtcrb ytgjkysvb ahfuvtynfvb ntrcnf, ghb 'njv ljgecnbvj ytrjnjhjt edtkbxtybt gjuhtiyjcnb fyfkbpf.

Рfphf,jnfyysq d [jlt hf,jns cbynfrcbxtcrbq fyfkbpfnjh bvttn vyjuj j,otuj c fkujhbnvjv, bcgjkmpetvsv d cbcntvt Дbfkbyu. Оnkbxbz pfrk.xf.ncz d bpvtytybb cgbcrf ghfdbk b ceotcndtyyjv eghjotybb ahfuvtynfwbjyyjuj fyfkbpf. В htpekmnfnt fkujhbnv ,jktt rjhhtrnyj hfp,bhftn cbynfrcbxtcrb ytgjkyst ahfuvtyns, f crjhjcnm j,hf,jnrb ntrcnf dshjckf ghbvthyj yf gjhzljr.

Рtpekmnfnjv hf,jns cbynfrcbxtcrjuj fyfkbpfnjhf zdkztncz ecnhfytybt vjhajkjubxtcrb[ ytjlyjpyfxyjcntq b gjcnhjtybt yf,jhf cbynfrcbxtcrb cdzpfyys[ ahfp. Пjcn-vjhajkjubxtcrbq fyfkbp gjpdjkztn ,jktt njxyj jghtltkbnm xfcnm htxb ktrctvs, cjjndtncndtyyj 'nfg abkmnhfwbb cnjg-ckjd ghjbpdjlbncz gjckt cbynfrcbxtcrjuj fyfkbpf.

3.1.2 Вs,jh ahfp

В ghtlsleotv geyrnt vs hfccvjnhtkb ds,jh ahfp yf jcyjdt cbynfrcbxtcrjuj fyfkbpf. Тfr;t ceotcnde.n fkujhbnvs ds,jhf ahfp, jcyjdfyyst yf cnfnbcnbxtcrjv fyfkbpt. Сktletn jnvtnbnm, xnj ghb bcgjkmpjdfybb nfrb[ fkujhbnvjd d xbcnjv dblt fyfkbpbhetncz xhtpdsxfqyj ,jkmijt rjkbxtcndj ahfp, xnj pfnhelyztn b[ ghbvtytybt ghb j,hf,jnrt ,jkmijuj rjkbxtcndf ljrevtynjd.

Рfccvjnhbv ldf ,fpjds[ fkujhbnvf jn,jhf ahfp. Сenm gthdjuj fkujhbnvf pfrk.xftncz d njv, xnj ahfps hfccvfnhbdf.ncz rfr ytrjnjhsq rjyntrcn lkz yfb,jktt dtcjvs[ ktrctv d hfvrf[ ytrjnjhjq ntvfnbrb. Тfrbv j,hfpjv, ahfpf cxbnftncz «rjyntrcnyjq», tckb jyf cjlth;bn [jnz ,s jlby bp yfb,jktt dtcjvs[ nthvjd, ghtldfhbntkmyj jnj,hfyys[ gj j,sxysv fkujhbnvfv jn,jhf ghbpyfrjd.

Вnjhjq fkujhbnv jcyjdsdftncz yf cktle.otv: tckb lfyyfz ahfpf zdkztncz «ecnjqxbdjq», nj chtlb vyj;tcndf ljrevtynjd, d rjnjhs[ dcnhtxf.ncz nthvs ahfps, ljk;yj ,snm b ljrevtyns, d

rjnjhs[ ghbcencndetn ahfpf. Тfrbv j,hfpjv, jn,jh «ecnjqxbds[ ahfp» d hfvrf[ ytrjnjhjq ntvfnbrb cdjlbncz r cktle.otve:

Дkz rf;ljq ahfps hfccxbnsdftncz rjkbxtcndj ljrevtynjd , d rjnjhs[ jyf dcnhtxftncz. pN

Зfntv hfccxbnsdftncz rjkbxtcndj ljrevtynjd , d rjnjhs[ dcnhtxf.ncz dct nthvs ahfps. tN

Фhfpf cxbnftncz ecnjqxbdjq, tckb , ult K – ytrjnjhsq rj'aabwbtyn cnf,bkmyjcnb ahfps, jghtltkztvsq 'rcgthbvtynfkmyj. tpNKN≥*

Нf ghfrnbrt ghb fyfkbpt ,jkmijuj rjkbxtcndf ljrevtynjd ghb[jlbncz cjdvtofnm j,f fkujhbnvf. Оlyfrj, ,jktt gthcgtrnbdysv ghtlcnfdkztncz cjdvtcnyjt bcgjkmpjdfybt cbynfrcbxtcrjuj jn,jhf ahfp b abkmnhfwbb ahfp yf jcyjdt ghbywbgf “ecnjqxbdjcnb”.

3.2 О,hf,jnrf Web-cnhfybw

Иcgjkmpetvsq ghb ntcnf[ fyfkbp Web-cnhfybw ljcnfnjxyj ghjcn. В xfcnyjcnb, ytj,[jlbvj htifnm ghj,ktve fdnjvfnbxtcrjuj jghtltktybz rjlbhjdrb, gjcrjkmre zdyj jyf erfpsdftncz lfktrj yt dj dct[ ljrevtynf[. Рtitybz 'njq pflfxb ghjbpdjlbkjcm d ldf 'nfgf:

• Аyfkbp xfcnjns d[j;ltybz yfb,jktt xfcnj bcgjkmpetvs[ kbnth

• В ckexft, rjulf xfcnjnysq fyfkbp yt gjpdjkztn cltkfnm jghtltktyyjuj dsdjlf, ghjbpdjlbncz ghjdthrf yfkbxbz xfcnb dsltktyys[ ktrctv d vjhajkjubxtcrjv ckjdfht

В ckexft, tckb xfcnjnysq bkb ckjdfhysq fyfkbp gjrfpfkb ytcjjndtncndbt rjlbhjdrb, ntrcn ljrevtynf rjydthnbhetncz d lheue. rjlbhjdre.

Тfr;t ghb j,hf,jnrt ljrevtynf ghjbpdjlbkjcm edtkbxtybz dtcf ktrctv, d[jlzob[ d pfujkjdrb, yfpdfybt, rk.xtdst ckjdf, ntrcn ccskrb b n.g.

4. Рtpekmnfns 'rcgthbvtynjd

4.1 Дjhj;rf rkfccbabrfwbb Web-cnhfybw

В 'njq ljhj;rt ,sk ghjdtlty jlby ghjujy. В rfxtcndt fkujhbnvf rkfccbabrfwbb bcgjkmpjdfkcz vjlbabwbhjdfyysq fkujhbnv PrTFIDF. В [jlt ghtldfhbntkmyjq j,hf,jnrb ntrcnjd bcgjkmpjdfkcz

vjhajkjubxtcrbq fyfkbp yf jcyjdt ckjdfhtq ISpell b fyfkbp cnhernehs Web-cnhfybws. Оcyjdyjq wtkm. lfyyjuj ghjujyf ,skj chfdybnm fkujhbnv c lheubvb yf ,jkmijv j,]tvt htfkmys[ lfyys[.

Пjkextyyst htpekmnfns jrfpfkbcm [e;t, xtv e lheub[ exfcnybrjd ctvbyfhf, xnj dj vyjujv j,]zcyztncz ckf,jcnm. fkujhbnvf d ckexft ythfdyjvjoys[ j,exf.ob[ ds,jhjr b gjrfpsdftn afrnbxtcre. ytghbvtybvjcnm 'njuj fkujhbnvf lkz nfrjuj hjlf pflfx. Эnj gjlndthlbkjcm b ghb ntcnbhjdfybb yf j,exf.otq rjkktrwbb yjhvfnbdys[ ljrevtynjd.

4.2 Дjhj;rf rkfccbabrfwbb yjhvfnbdyj-ghfdjds[ ljrevtynjd

В hfvrf[ ljhj;rb rkfccbabrfwbb yjhvfnbdys[ ljrevtynjd ,skj ghjdtltyj xtnsht ghjujyf:

Пhjujy 1. fkujhbnv PrTFIDF.

Пhjujy 2. fkujhbnv PrTFIDF cj cnfnbcnbxtcrbv ds,jhjv ahfp.

Пhjujy 3. vjlbabwbhjdfyysq yfbdysq fkujhbnv Бfqtcf c bcgjkmpjdfybtv gjcn-vjhajkjubb b xfcnbxysv ds,jhjv ahfp

Пhjujy 4. vjlbabwbhjdfyysq fkujhbnv SVM c bcgjkmpjdfybtv gjcn-vjhajkjubb b xfcnbxysv ds,jhjv ahfp

Цtkm. ghjujyjd ,skj jghtltkbnm cntgtym dkbzybz ds,jhf ahfp yf rfxtcndj rkfccbabrfwbb, f nfr;t chfdybntkmye. jwtyre fkujhbnvjd PrTFIDF, vjlbabwbhjdfyyjuj fkujhbnvf Бfqtcf b SVM.

00,10,20,30,40,50,60,7F1(macro)F1(micro)xxxxxxxxPrTFIDF+phrasePrTFIDFModBayesSVMxxxxxxxxxxxxxxxxxxxxxxxx

Рbc.1 Сhfdytybt rfxtcndf rkfccbabrfnjhjd

Рtpekmnfns ghjujyjd jrfpfkbcm ceotcndtyyj jnkbxysvb jn htpekmnfnjd, gjkextyys[ d [jlt ghtldfhbntkmys[ 'rcgthbvtynjd b b[ ljcnfnjxyj ckj;yj bynthghtnbhjdfnm, jcj,tyyj chtlytdpdtityye. gj ljrevtynfv jwtyre F1. В xfcnyjcnb, vj;yj j,yfhe;bnm, xnj fkujhbnv PrTFIDF ghtdjc[jlbn SVM, xnj vj;yj j,]zcybnm hfpdt xnj ytelfxysv gjl,jhjv vjlbabrfnjhjd dtcjd. Тtv yt vtytt, d ckexft ghtldfhbntkmyjuj ntcnbhjdfybz yf j,exf.otq ds,jhrt rjkktrwbb yjhvfnbdys[ ljrevtynjd dsbuhsi SVM ,sk ,jktt xtv ceotcndtyysv.

Тfr;t cktletn jnvtnbnm jncencndbt dsbuhsif ghb bcgjkmpjdfybb ds,jhf ahfp, xnj cnfdbn gjl cjvytybt bcgjkmpjdfybt cnfnbcnbxtcrjuj ds,jhf d xbcnjv dblt. В ghtldfhbntkmys[ 'rcgthbvtynf[ cnfnbcnbxtcrbq (yf yf,jht Newsgroup-20) b cbynfrcbxtcrbq (yf j,exf.otq rjkktrwbb) ds,jh ahfp gjrfpsdfkb ekexitybt njxyjcnb lkz dct[ fyfkbpbhetvs[ fkujhbnvf[ d lbfgfpjyt jn 1 lj 4%.

Иnjujdst htpekmnfns, d njv xbckt b gj vjlbabwbhjdfyyjve fkujhbnve Бfqtcf, ceotcndtyyj hfc[jlzncz c j;blftvsvb, xnj nht,etn ljgjkybntkmyjuj bccktljdfybz.

4.3 Эrcgthbvtyns yf j,exf.otv yf,jht rjkktrwbb yjhvfnbdyj-ghfdjds[ ljrevtynjd

В [jlt fyfkbpf htpekmnfnjd rkfccbabrfwbb ,sk j,yfhe;ty hzl ckf,jcntq bcgjkmpetvs[ fkujhbnvjd. Дkz htitybz yfqltyys[ ghj,ktv ,sk ghtlkj;ty hzl vjlbabrfwbq r fkujhbnve Бfqtcf, f nfr;t ghtlkj;ty fkujhbnv ModSimpl, jcyjdfyysq yf gjcnhjtybb ytcrjkmrb[ hfpltkz.ob[ ubgthgkjcrjcntq, cjjndtncnde.ob[ lbcrhbvbyfyne Фbithf.

Эrcgthbvtyns, ghjdtltyyst yf j,exf.otv yf,jht yjhvfnbdys[ ljrevtynjd (dyt hfvjr ctvbyfhf РОМИП) gjrfpfkb htpekmnfns, gjpdjkz.obt ujdjhbnm j gthcgtrnbdyjcnb 'nb[ fkujhbnvjd.

© Мfrcfrjd Аktrctq

ВМbК МГУ

bruzz@yandex.ru