ÀÁÐÀÊÀÄÀÁÐÀ (Òîæå ñàìîå íî â ÷èòàåìîì âèäå)
Äfyyfz cnfnmz gjcdzotyf ldev jcyjdysv ghj,ktvfv he,hbrfwbb ntrcnjd: ds,jhe fkujhbnvf rkfccbabrfwbb b cgjcj,fv ghtldfhbntkmyjq j,hf,jnrb ntrcnf. Íf jcyjdt 'rcgthbvtynjd d hfvrf[ ctvbyfhf ÐÎÌÈÏ’2005 ,sk ghjdtlty chfdybntkmysq fyfkbp hfccvfnhbdftvs[ gjl[jljd b ghtlkj;tys cgjcj,s htitybz j,yfhe;tyys[ ghj,ktv. 1. Âdtltybt
Äfyyfz hf,jnf ghjdjlbncz d hfvrf[ bccktljdfybz cgjcj,jd j,tcgtxtybz gthbjlbxtcrjuj ntvfnbxtcrjuj gjbcrf, xnj drk.xftn d ct,z b hfphf,jnre nfrjuj hjlf cbcntvs. Îlybv bp jcyjdys[ 'nfgjd hf,jns 'njq cbcntvs zdkztncz 'nfg rkfccbabrfwbb ntrcnjd. Ïjcrjkmre rfxtcndj he,hbrfwbb dj vyjujv jghtltkztn rfxtcndj bnjujdjuj htpekmnfnf, bccktljdfybt fkujhbnvjd rkfccbabrfwbb b cgjcj,jd ghtldfhbntkmyjq j,hf,jnrb ljrevtynjd, zdkztncz df;ytqitq pflfxtq.
Àkujhbnvs rkfccbabrfwbb hf,jnf.n c ytrjnjhjq vfntvfnbxtcrjq vjltkm. ghtlcnfdktybz 'rptvgkzhf (d lfyyjv ckexft – ntrcnjdjuj ljrevtynf). Ífb,jktt hfcghjcnhfytyyjq vjltkm. zdkztncz ghtlcnfdktybt d dblt yf,jhf ghbpyfrjd, rjnjhjt b ,eltn hfccvjnhtyj d lfkmytqitv. Îghtltktybt ghbpyfrjd b cjgjcnfdktybt bv dtcjd zdkztncz ceotcndtyyj ytajhvfkmysv ifujv b dj vyjujv dkbztn yf htpekmnfn rkfccbabrfwbb. Ïj 'njq ghbxbyt 'nfg ghtldfhbntkmyjq j,hf,jnrb ljrevtynf hfccvfnhbdftncz jnltkmyj.
2. Ðfccvfnhbdftvst fkujhbnvs
 [jlt 'rcgthbvtynjd ,skb hfccvjnhtys nhb ghtlcnfdbntkz ctvtqcndf kbytqys[ fkujhbnvjd: vtnjl jgjhys[ dtrnjhjd (SVM) [1], fkujhbnv PrTFIDF[2] b vjlbabwbhjdfyysq yfbdysq fkujhbnv Áfqtcf[3].
Àkujhbnvs PrTFIDF b yfbdysq fkujhbnv Áfqtcf jcyjdfys yf cnfnbcnbxtcrjq vjltkb b bvt.n vyjuj j,otuj. Äfyyst fkujhbnvs ghtlcnfdkz.n bynthtc, gjcrjkmre jyb [jhjij vfcinf,bhetvs b j,kflf.n dscjrjq ghjbpdjlbntkmyjcnm.. Èpdtcnysv b[ ytljcnfnrjv zdkztncz chfdybntkmyj ybprfz njxyjcnm rkfccbabrfwbb, jcj,tyyj d ckexft ,byfhyjq rkfccbabrfwbb. Àkujhbnv PrTFIDF hfccvfnhbdftncz d rfxtcndt ,fpjdjuj fkujhbnvf gj hzle ghbxby:
• Ýrcgthbvtynfkmyj[2] gjrfpfyf ,jktt dscjrfz njxyjcnm rkfccbabrfwbb gj chfdytyb. c yfbdysv Áfqtcjv b TFIDF[4]. Òfr;t 'nj ,skj gjlndth;ltyj b d lheub[ 'rcgthbvtynf[, ds[jlzob[ pf hfvrb lfyyjq cnfnmb.
• Äfyysq fkujhbnv ghbvtybv lkz fyfkbpf ,jkmijuj rjkbxtcndf ljrevtynjd b gjpdjkztn bcgjkmpjdfnm ,jkmijt rjkbxtcndj ghbpyfrjd. Ýnj df;yj, gjcrjkmre fkujhbnv gkfybhetncz ghbvtyznm lkz j,hf,jnrb ,jkmib[ j,]tvjd lfyys[.
Àkujhbnv Áfqtcf d gjcktlytt dhtvz jwtybdftncz rfr chfdybntkmyj ybprjrfxtcndtyysq fkujhbnv. Îcyjdysvb ghbxbyfvb zdkz.ncz ghj,ktvs, cdzpfyyst c ghbywbgjv ytpfdbcbvjcnb ghbpyfrjd b ytrjhhtrnyjq jwtyrjq fghbjhyjq dthjznyjcnb d ckexft ceotcndtyyj ythfdyjvjoys[ j,exf.ob[ ds,jhjr. Ïhtlkj;bd hzl 'vgbhbxtcrb[ vjlbabrfwbq fkujhbnvf bkb lj,fdbd ljgjkybntkmyst ghbpyfrb, yfghbvth, yf jcyjdt ds,jhf ahfp d ljrevtynt, vj;yj gjgsnfnmcz htibnm ceotcnde.obt ghj,ktvs b gjkexbnm ,jktt rfxtcndtyysq fkujhbnv, cj[hfybd tuj ghjcnjne, ghjbpdjlbntkmyjcnm b vfcinf,bhetvjcnm.
Ðtpekmnfns rkfccbabrfwbb vtnjljv jgjhys[ dtrnjhjd d gjcktlytt dhtvz jwtybdf.ncz[5] rfr kexibt bkb jlyb bp kexib[. Îlyfrj, crjhjcnm j,extybz lfyyjuj fkujhbnvf chfdybntkmyj ybprf (O(|D|a, ult a>1,2[5])) b jy nht,etn ,jkmijuj j,]tvf gfvznb, xnj cyb;ftn tuj vfcinf,bhetvjcnm. Òtv yt vtytt, lfyysq fkujhbnv vj;yj bcgjkmpjdfnm d rfxtcndt 'nfkjyf c njxrb phtybz rfxtcndf rkfccbabrfwbb. Òfr;t ,skb ghtlkj;tys vjlbabrfwbb jwtyrb dtcjd ghbpyfrjd, rjnjhst ,elen hfccvjnhtys gjplytt.
Òfrbv j,hfpjv, nht,jdfybz r fkujhbnve rkfccbabrfwbb d hfvrf[ htiftvjq pflfxb vj;yj cajhvekbhjdfnm cktle.obv j,hfpjv:
1. Êfxtcndj rkfccbabrfwbb ljk;yj ,snm chfdybvj c rfxtcndjv vtnjlf jgjhys[ dtrnjhjd.
2. Àkujhbnv ljk;ty j,kflfnm ybprjq dsxbckbntkmyjq ckj;yjcnm. b zdkztncz [jhjij vfcinf,bhetvsv.
Ðfccvjnhbv ghtlkfuftvst vjlbabrfwbb r ceotcnde.obv fkujhbnvfv.
2.1 Ìtnjlbrf ghtldfhbntkmyjq jwtyrb vjlbabrfwbq
Ìjlbabrfwbb fkujhbnvjd nht,e.n 'rcgthbvtynfkmyjq ghjdthrb lkz jwtyrb b[ dkbzybz yf fkujhbnv. Â hfvrf[ lfyyjq hf,jns ghtldfhbntkmyfz jwtyrf ghjbpdjlbkfcm yf lde[ ntcnjds[ yf,jhf[: Newsgroup-20[6] b j,exf.otq rjkktrwbb yjhvfnbdys[ ljrevtynjd. Äkz dnjhjq rjkktrwbb d rfxtcndt j,exf.otq ds,jhrb ,skj ds,hfyj 40% ljrevtynjd ckexfqysv j,hfpjv, jcnfkmyst ljrevtyns bcgjkmpjdfkbcm lkz jwtyrb njxyjcnb rkfccbabrfwbb. Ânjhfz ds,jhrf bynthtcyf cbkmyjq ythfdyjvthyjcnm. hfcghtltktybz ljrevtynjd gj rkfccfv b ,jkmibv rjkbxtcndjv rkfccjd.
2.2 Ìjlbabrfwbb yfbdyjuj fkujhbnvf Áfqtcf
Ïhfdbkj jghtltktybz rkfccf lkz ljrevtynf d fkujhbnvt Áfqtcf vj;yj ghtlcnfdbnm cktle.obv j,hfpjv:
]log)([log(maxarg)(Σ∈+=dwCwwCpfCpdC,
ult - rjkbxtcndj d[j;ltybq ktrctvs w d ljrevtyn, wf)|(CwppCw=
Äkz ,jhm,s c ytrjhhtrnysv jghtltktybtv fghbjhyjq eckjdyjq dthjznyjcnb ghbpyfrjd d ckexft ythfdyjvjoys[ j,exf.ob[ ds,jhjr, ghtlkfuftncz bcgjkmpjdfnm gfhflbuve rkfccf-ljgjkytybz, nj tcnm dvtcnj dthjznyjcnb ghbyflkt;yjcnb ktrctvs rkfcce jwtybdfnm dthjznyjcnm ghbyflkt;yjcnb ktrctvs rkfcce-ljgjkytyb. C’ (cktletn extcnm, xnj p(w|C) ~ 1-p(w|C’)).Ècgjkmpez ghbywbg cukf;bdfybz gfhfvtnhjd gj Ëfgkfce, gjkexftv cktle.ott ghfdbkj:
])||1log()([log(maxarg)(Σ∈++−=dwCCwwCVNNfCpdC
ult CwN- rjkbxtcndj ktrctv dj dct[ rkfccf[ rhjvt lfyyjuj, CN-j,ott rjkbxtcndj ktrctv d rkfcct-ljgjkytybb, -hfpvthyjcnm ckjdfhz ktrctv. ||V
Ñktletn jnvtnbnm, xnj lfyyfz 'dhbcnbrf hf,jnftn njkmrj d njv ckexft, tckb rjkbxtcndj rkfccjd N >> 2.
Äkz lfkmytqituj ekexitybz rfxtcndf rkfccbabrfwbb ghtlkfuf.ncz cktle.obt ghbtvs:
• Ëjufhbavbxtcrjt cukf;bdfybt xfcnjns ghbpyfrjd
• Íjhvfkbpfwbz dtcjd ghbpyfrjd d ljrevtynt gj tuj lkbyt
• Ècgjkmpjdfybt bydthcyjq xfcnjns ghbpyfrf (IDF b IDF’[2])
• Íjhvfkbpfwbz kjufhbavjd dtcjd ghbpyfrjd (log(pCw))
Ïhtldfhbntkmyst 'rcgthbvtyns gjrfpfkb ekexitybt njxyjcnb rkfccbabrfwbb ghb drk.xtybb dct[ 'dhbcnbr, rhjvt kjufhbavbxtcrjuj cukf;bdfybz b bcgjkmpjdfybz bydthcyjq xfcnjns. Â bnjut gthtl jrjyxfntkmysv ghjujyjv fkujhbnvf (lfktt ModBayes) e[elif.obt rfxtcndj 'dhbcnbrb ,skb jnrk.xtys. Òjxyjcnm fkujhbnvf ghb ghtldfhbntkmyjv ntcnbhjdfybb jrfpfkfcm chfdybvjq c njxyjcnm. SVM, ghb 'njv njxyjcnm ,fpjdjuj fkujhbnvf Áfqtcf ,skf ,kbprf r yek..
2.3 Ìjlbabrfwbb fkujhbnvf SVM
Ðfccvfnhbdftvfz vjlbabrfwbz fkujhbnvf cdjlbncz r nhbdbfkmyjve 'vgbhbxtcrjve bpvtytyb. jwtyrb dtcf ghbpyfrjd. Èpyfxfkmyst ghtlgjcskrb j,eckjdktys cktle.obv:
• Ëtrctvs c dscjrjq bydthcyjq xfcnjnjq djpvj;yj ,jktt pyfxbvs, b cjjndtncndtyyj ljk;ys bvtnm ,jkmibq dtc, fyfkjubxyj ghtlgjkj;tybzv fkujhbnvf TFIDF.
• Åckb ktrctvf xfcnj dcnhtxftncz d ljrevtynf[ jlyjuj rkfccf, yj htlrj d ljrevtynf[ lheujuj, nj 'nf ktrctvf nfr;t djpvj;yj ,jktt pyfxbvf, xtv ktrctvf, dcnhtxf.ofzcz d vfkjv rjkbxtcndt ljrevtynjd, yj dj vyjub[ rkfccf[. Â rfxtcndt ghbvthf vj;yj ghbdtcnb ldt cbnefwbb: ktrctvf dcnhtxftncz d ltcznb ljrevtynf[ jlyjuj rkfccf, f lheufz gj ldf hfpf d j,jb[ rkfccf[. Ñ njxrb phtybz bydthcyjq xfcnjns dnjhfz ktrctvf ,eltn bvtnm ,jkmibq dtc, yj afrnbxtcrb gthdfz ujhfplj ,jktt pyfxbvf lkz rfxtcndtyyjuj hfpltktybz lde[ rkfccjd.
Òfrbv j,hfpjv ,sk ghtlkj;ty cktle.obq vjlbabrfnjh dtcf ktrctvs:
'*)',(max'IDFCwTFCC∈, ult ΣΣ∈∈=CCFwCwTFCwTFDIDF'')','()',(||'
 [jlt ghtldfhbntkmys[ 'rcgthbvtynjd yf ntcnjdjq rjkktrwbb Newsgroup-20 ghbvtytybt 'njq 'dhbcnbrb ghbdtkj r yt,jkmijve edtkbxtyb. njxyjcnb rkfccbabrfwbb. Ïhb ghjujyt fkujhbnvf SVM yf rjkktrwbb yjhvfnbdys[ ljrevtynjd 'nf 'dhbcnbrf ,skf drk.xtyf.
3. Ïhtldfhbntkmyfz j,hf,jnrf ljrevtynjd
Çflfxtq 'nfgf ghtldfhbntkmyjq j,hf,jnrb ljrevtynjd zdkztncz dsltktybt ghbpyfrjd ljrevtynf b cjgjcnfdktybz bv dtcjd. Â ghjcntqitv ckexft vekmnbyjvbfkmyjq vjltkb yf,jhjv ghbpyfrjd ljrevtynf ,eltn cjlth;fobqcz d ytv yf,jh ktrctv, f d rfxtcndt dtcf bcgjkmpetncz rjkbxtcndj d[j;ltybq ktrctvs d ljrevtyn.
Ítljcnfnrjv nfrjuj gjl[jlf zdkztncz nj, xnj ghfrnbxtcrb yt exbnsdf.ncz jcj,tyyjcnb tcntcndtyyjuj zpsrf, f nfr;t cnhernehbhjdfyyjcnm ljrevtynf b cdzpb vt;le ljrevtynfvb d ckexft Web-cnhfybw.
3.1 Î,hf,jnrf ntrcnjd yf tcntcndtyyjv zpsrt
Ïhb j,hf,jnrt ntrcnf vj;yj dsltkbnm ytcrjkmrj 'nfgjd:
1. Ëtrcbxtcrbq fyfkbp
2. Ìjhajkjubxtcrbq fyfkbp
3. Ñbynfrcbxtcrbq b gjcn-vjhajkjubxtcrbq fyfkbp
4. Âsltktybt ahfp (n-uhfvv)
5. Ócnhfytybt cnjg-ckjd
Ïthdst ldf 'nfgf ljcnfnjxyj jxtdblys: pflfxtq gthdjuj zdkztncz dsltktybt ktrctv, f dnjhjq yf jcyjdt yf,jhf ghfdbk b dyenhtyytuj ckjdfhz cjgjcnfdkztn rf;ljq ktrctvt yf,jh djpvj;ys[ ckjdjjcyjd c b[ uhfvvfnbxtcrbvb [fhfrnthbcnbrfvb.
Ècgjkmpjdfybt cbynfrcbxtcrjuj fyfkbpf gjpdjkztn hfphtibnm pyfxbntkmye. xfcnm ckexftd jvjybvbb. Òfr;t cbynfrcbxtcrbq fyfkbp vj;tn gjpdjkbnm j,tcgtxbnm ,jktt njxye. abkmnhfwb. cnjg-ckjd b gjcnhjtybt ahfp yf jcyjdt cbynfrcbxtcrb cdzpfyys[ ktrctv, xnj ceotcndtyyj cjrhfoftn b[ rjkbxtcndj gj chfdytyb. c gjkysv gtht,jhjv cjctltq ktrctvs.
3.1.1 Ñbynfrcbxtcrbq fyfkbp
Ïhj,ktvfvb ceotcnde.ob[ htitybq d j,kfcnb cbynfrcbxtcrjuj fyfkbpf (yfghbvth, LinkParser [7], Äbfkbyu[8]) zdkz.ncz ljcnfnjxyj ybprfz crjhjcnm j,hf,jnrb ntrcnf b xedcndbntkmyjcnm r ytrjhhtrnysv cbynfrcbxtcrbv rjycnherwbzv. Ýnb ghj,ktvs cktle.n bp j,kfcntq ghbvtytybz 'nb[ htitybq – ghjdthrf ghfdjgbcfybz b vfibyysq gthtdjl. Â pflfxt rkfccbabrfwbb ntrcnjd nht,jdfybz r cbynfrcbxtcrjve fyfkbpe ytcrjkmrj lheubt: dscjrfz crjhjcnm j,hf,jnrb ntrcnjd b hf,jnf c cbynfrcbxtcrb ytgjkysvb ahfuvtynfvb ntrcnf, ghb 'njv ljgecnbvj ytrjnjhjt edtkbxtybt gjuhtiyjcnb fyfkbpf.
Ðfphf,jnfyysq d [jlt hf,jns cbynfrcbxtcrbq fyfkbpfnjh bvttn vyjuj j,otuj c fkujhbnvjv, bcgjkmpetvsv d cbcntvt Äbfkbyu. Înkbxbz pfrk.xf.ncz d bpvtytybb cgbcrf ghfdbk b ceotcndtyyjv eghjotybb ahfuvtynfwbjyyjuj fyfkbpf. Â htpekmnfnt fkujhbnv ,jktt rjhhtrnyj hfp,bhftn cbynfrcbxtcrb ytgjkyst ahfuvtyns, f crjhjcnm j,hf,jnrb ntrcnf dshjckf ghbvthyj yf gjhzljr.
Ðtpekmnfnjv hf,jns cbynfrcbxtcrjuj fyfkbpfnjhf zdkztncz ecnhfytybt vjhajkjubxtcrb[ ytjlyjpyfxyjcntq b gjcnhjtybt yf,jhf cbynfrcbxtcrb cdzpfyys[ ahfp. Ïjcn-vjhajkjubxtcrbq fyfkbp gjpdjkztn ,jktt njxyj jghtltkbnm xfcnm htxb ktrctvs, cjjndtncndtyyj 'nfg abkmnhfwbb cnjg-ckjd ghjbpdjlbncz gjckt cbynfrcbxtcrjuj fyfkbpf.
3.1.2 Âs,jh ahfp
 ghtlsleotv geyrnt vs hfccvjnhtkb ds,jh ahfp yf jcyjdt cbynfrcbxtcrjuj fyfkbpf. Òfr;t ceotcnde.n fkujhbnvs ds,jhf ahfp, jcyjdfyyst yf cnfnbcnbxtcrjv fyfkbpt. Ñktletn jnvtnbnm, xnj ghb bcgjkmpjdfybb nfrb[ fkujhbnvjd d xbcnjv dblt fyfkbpbhetncz xhtpdsxfqyj ,jkmijt rjkbxtcndj ahfp, xnj pfnhelyztn b[ ghbvtytybt ghb j,hf,jnrt ,jkmijuj rjkbxtcndf ljrevtynjd.
Ðfccvjnhbv ldf ,fpjds[ fkujhbnvf jn,jhf ahfp. Ñenm gthdjuj fkujhbnvf pfrk.xftncz d njv, xnj ahfps hfccvfnhbdf.ncz rfr ytrjnjhsq rjyntrcn lkz yfb,jktt dtcjvs[ ktrctv d hfvrf[ ytrjnjhjq ntvfnbrb. Òfrbv j,hfpjv, ahfpf cxbnftncz «rjyntrcnyjq», tckb jyf cjlth;bn [jnz ,s jlby bp yfb,jktt dtcjvs[ nthvjd, ghtldfhbntkmyj jnj,hfyys[ gj j,sxysv fkujhbnvfv jn,jhf ghbpyfrjd.
Ânjhjq fkujhbnv jcyjdsdftncz yf cktle.otv: tckb lfyyfz ahfpf zdkztncz «ecnjqxbdjq», nj chtlb vyj;tcndf ljrevtynjd, d rjnjhs[ dcnhtxf.ncz nthvs ahfps, ljk;yj ,snm b ljrevtyns, d
rjnjhs[ ghbcencndetn ahfpf. Òfrbv j,hfpjv, jn,jh «ecnjqxbds[ ahfp» d hfvrf[ ytrjnjhjq ntvfnbrb cdjlbncz r cktle.otve:
Äkz rf;ljq ahfps hfccxbnsdftncz rjkbxtcndj ljrevtynjd , d rjnjhs[ jyf dcnhtxftncz. pN
Çfntv hfccxbnsdftncz rjkbxtcndj ljrevtynjd , d rjnjhs[ dcnhtxf.ncz dct nthvs ahfps. tN
Ôhfpf cxbnftncz ecnjqxbdjq, tckb , ult K – ytrjnjhsq rj'aabwbtyn cnf,bkmyjcnb ahfps, jghtltkztvsq 'rcgthbvtynfkmyj. tpNKN≥*
Íf ghfrnbrt ghb fyfkbpt ,jkmijuj rjkbxtcndf ljrevtynjd ghb[jlbncz cjdvtofnm j,f fkujhbnvf. Îlyfrj, ,jktt gthcgtrnbdysv ghtlcnfdkztncz cjdvtcnyjt bcgjkmpjdfybt cbynfrcbxtcrjuj jn,jhf ahfp b abkmnhfwbb ahfp yf jcyjdt ghbywbgf “ecnjqxbdjcnb”.
3.2 Î,hf,jnrf Web-cnhfybw
Ècgjkmpetvsq ghb ntcnf[ fyfkbp Web-cnhfybw ljcnfnjxyj ghjcn. Â xfcnyjcnb, ytj,[jlbvj htifnm ghj,ktve fdnjvfnbxtcrjuj jghtltktybz rjlbhjdrb, gjcrjkmre zdyj jyf erfpsdftncz lfktrj yt dj dct[ ljrevtynf[. Ðtitybz 'njq pflfxb ghjbpdjlbkjcm d ldf 'nfgf:
• Àyfkbp xfcnjns d[j;ltybz yfb,jktt xfcnj bcgjkmpetvs[ kbnth
• Â ckexft, rjulf xfcnjnysq fyfkbp yt gjpdjkztn cltkfnm jghtltktyyjuj dsdjlf, ghjbpdjlbncz ghjdthrf yfkbxbz xfcnb dsltktyys[ ktrctv d vjhajkjubxtcrjv ckjdfht
 ckexft, tckb xfcnjnysq bkb ckjdfhysq fyfkbp gjrfpfkb ytcjjndtncndbt rjlbhjdrb, ntrcn ljrevtynf rjydthnbhetncz d lheue. rjlbhjdre.
Òfr;t ghb j,hf,jnrt ljrevtynf ghjbpdjlbkjcm edtkbxtybz dtcf ktrctv, d[jlzob[ d pfujkjdrb, yfpdfybt, rk.xtdst ckjdf, ntrcn ccskrb b n.g.
4. Ðtpekmnfns 'rcgthbvtynjd
4.1 Äjhj;rf rkfccbabrfwbb Web-cnhfybw
 'njq ljhj;rt ,sk ghjdtlty jlby ghjujy.  rfxtcndt fkujhbnvf rkfccbabrfwbb bcgjkmpjdfkcz vjlbabwbhjdfyysq fkujhbnv PrTFIDF.  [jlt ghtldfhbntkmyjq j,hf,jnrb ntrcnjd bcgjkmpjdfkcz
vjhajkjubxtcrbq fyfkbp yf jcyjdt ckjdfhtq ISpell b fyfkbp cnhernehs Web-cnhfybws. Îcyjdyjq wtkm. lfyyjuj ghjujyf ,skj chfdybnm fkujhbnv c lheubvb yf ,jkmijv j,]tvt htfkmys[ lfyys[.
Ïjkextyyst htpekmnfns jrfpfkbcm [e;t, xtv e lheub[ exfcnybrjd ctvbyfhf, xnj dj vyjujv j,]zcyztncz ckf,jcnm. fkujhbnvf d ckexft ythfdyjvjoys[ j,exf.ob[ ds,jhjr b gjrfpsdftn afrnbxtcre. ytghbvtybvjcnm 'njuj fkujhbnvf lkz nfrjuj hjlf pflfx. Ýnj gjlndthlbkjcm b ghb ntcnbhjdfybb yf j,exf.otq rjkktrwbb yjhvfnbdys[ ljrevtynjd.
4.2 Äjhj;rf rkfccbabrfwbb yjhvfnbdyj-ghfdjds[ ljrevtynjd
 hfvrf[ ljhj;rb rkfccbabrfwbb yjhvfnbdys[ ljrevtynjd ,skj ghjdtltyj xtnsht ghjujyf:
Ïhjujy 1. fkujhbnv PrTFIDF.
Ïhjujy 2. fkujhbnv PrTFIDF cj cnfnbcnbxtcrbv ds,jhjv ahfp.
Ïhjujy 3. vjlbabwbhjdfyysq yfbdysq fkujhbnv Áfqtcf c bcgjkmpjdfybtv gjcn-vjhajkjubb b xfcnbxysv ds,jhjv ahfp
Ïhjujy 4. vjlbabwbhjdfyysq fkujhbnv SVM c bcgjkmpjdfybtv gjcn-vjhajkjubb b xfcnbxysv ds,jhjv ahfp
Ötkm. ghjujyjd ,skj jghtltkbnm cntgtym dkbzybz ds,jhf ahfp yf rfxtcndj rkfccbabrfwbb, f nfr;t chfdybntkmye. jwtyre fkujhbnvjd PrTFIDF, vjlbabwbhjdfyyjuj fkujhbnvf Áfqtcf b SVM.
00,10,20,30,40,50,60,7F1(macro)F1(micro)xxxxxxxxPrTFIDF+phrasePrTFIDFModBayesSVMxxxxxxxxxxxxxxxxxxxxxxxx
Ðbc.1 Ñhfdytybt rfxtcndf rkfccbabrfnjhjd
Ðtpekmnfns ghjujyjd jrfpfkbcm ceotcndtyyj jnkbxysvb jn htpekmnfnjd, gjkextyys[ d [jlt ghtldfhbntkmys[ 'rcgthbvtynjd b b[ ljcnfnjxyj ckj;yj bynthghtnbhjdfnm, jcj,tyyj chtlytdpdtityye. gj ljrevtynfv jwtyre F1. Â xfcnyjcnb, vj;yj j,yfhe;bnm, xnj fkujhbnv PrTFIDF ghtdjc[jlbn SVM, xnj vj;yj j,]zcybnm hfpdt xnj ytelfxysv gjl,jhjv vjlbabrfnjhjd dtcjd. Òtv yt vtytt, d ckexft ghtldfhbntkmyjuj ntcnbhjdfybz yf j,exf.otq ds,jhrt rjkktrwbb yjhvfnbdys[ ljrevtynjd dsbuhsi SVM ,sk ,jktt xtv ceotcndtyysv.
Òfr;t cktletn jnvtnbnm jncencndbt dsbuhsif ghb bcgjkmpjdfybb ds,jhf ahfp, xnj cnfdbn gjl cjvytybt bcgjkmpjdfybt cnfnbcnbxtcrjuj ds,jhf d xbcnjv dblt. Â ghtldfhbntkmys[ 'rcgthbvtynf[ cnfnbcnbxtcrbq (yf yf,jht Newsgroup-20) b cbynfrcbxtcrbq (yf j,exf.otq rjkktrwbb) ds,jh ahfp gjrfpsdfkb ekexitybt njxyjcnb lkz dct[ fyfkbpbhetvs[ fkujhbnvf[ d lbfgfpjyt jn 1 lj 4%.
Ènjujdst htpekmnfns, d njv xbckt b gj vjlbabwbhjdfyyjve fkujhbnve Áfqtcf, ceotcndtyyj hfc[jlzncz c j;blftvsvb, xnj nht,etn ljgjkybntkmyjuj bccktljdfybz.
4.3 Ýrcgthbvtyns yf j,exf.otv yf,jht rjkktrwbb yjhvfnbdyj-ghfdjds[ ljrevtynjd
 [jlt fyfkbpf htpekmnfnjd rkfccbabrfwbb ,sk j,yfhe;ty hzl ckf,jcntq bcgjkmpetvs[ fkujhbnvjd. Äkz htitybz yfqltyys[ ghj,ktv ,sk ghtlkj;ty hzl vjlbabrfwbq r fkujhbnve Áfqtcf, f nfr;t ghtlkj;ty fkujhbnv ModSimpl, jcyjdfyysq yf gjcnhjtybb ytcrjkmrb[ hfpltkz.ob[ ubgthgkjcrjcntq, cjjndtncnde.ob[ lbcrhbvbyfyne Ôbithf.
Ýrcgthbvtyns, ghjdtltyyst yf j,exf.otv yf,jht yjhvfnbdys[ ljrevtynjd (dyt hfvjr ctvbyfhf ÐÎÌÈÏ) gjrfpfkb htpekmnfns, gjpdjkz.obt ujdjhbnm j gthcgtrnbdyjcnb 'nb[ fkujhbnvjd.
NB |
PrTFIDF |
ModBayes |
ModSimpl |
SVM |
njxyjcnm |
< 10% |
< 10% |
45,46% |
44,54% |
47,83% | <!--[if !supportMisalignedColumns]-->
|
|
|
|
|
|
|
|
|
| <!--[endif]-->
© Ìfrcfrjd Àktrctq
ÂÌbÊ ÌÃÓ
bruzz@yandex.ru |