Mene ne wani crawler? crawler kayan aiki "yandex" da kuma Google

Kowace rana a kan Internet akwai wata babbar adadin sabon kayan don ƙirƙirar wani website updated da haihuwa shafukan yanar gizo, aika hotuna da kuma bidiyo. Ba tare da boye daga search engines ba a iya samu a cikin World Wide Web, babu wani daga wadannan takardu. Zabi kamar robotic shirye-shirye a kowace lokaci ba ya wanzu. Mene ne wani search robot, me ya sa ka bukatar shi da kuma yadda za a yi aiki?

Mene ne wani search robot

Crawler site (search engine) - shi ne wani atomatik shirin cewa zai iya ziyarci miliyoyin shafukan yanar gizo, da sauri kewayawa ta hanyar internet ba tare da wani ma'aikaci baki. Bots ne kullum scan sarari na World Wide Web, da gano sababbin shafukan yanar gizo da kuma a kai a kai ziyarta riga fihirisa. Sauran sunayen for yanar gizo crawlers gizo-gizo, crawlers, Bots.

Me ne search engine gizo-gizo

Babban ayyuka da cewa yin search engine gizo-gizo - shafukan yanar gizo fihirisa, kazalika da rubutu, hotuna, sauti da bidiyo fayiloli da suke a kan su. Bots duba nassoshi, madubi shafukan (kofe) da kuma updates. A mutummutumi kuma yi HTML-code iko ga SHEDAN matsayin na Duniya Organization, wanda tasowa da kuma aiwatar da fasaha matsayin na World Wide Web.

Mene ne Indexing, kuma me ya sa shi ake bukata

Indexing - An, a gaskiya, shi ne aiwatar da ziyartar wani musamman shafin yanar gizo ta search injuna. A shirin yin sikanin da rubutu a kan wannan shafin, hotuna, bidiyo, outbound links, sa'an nan da page bayyana a cikin search results. A wasu lokuta, da shafin ba za a iya leka ta atomatik, sa'an nan shi za a iya kara wa search engine da hannu kula da shafukan yanar. Yawanci, wannan na faruwa a cikin rashi na waje links to da wani musamman (sau da yawa kawai kwanan halitta) page.

Ta yaya bincika engine gizo-gizo

Kowane search engine na da bot tare da Google search robot iya bambanta muhimmanci bisa ga inji aiki a kan irin wannan shirin, "yandex" ko sauran tsarin.

A general, wani robot aiki manufa ne kamar haka: da shirin "zo" a shafin da waje links daga main page, "karanta" Web hanya (ciki har da wadanda neman sama da cewa ba ya ganin mai amfani). Jirgin ruwan ne yadda za a kewaya tsakanin shafukan da wani site da kuma matsa wa wasu.

A shirin za a zabi wanda site to index? More sau da yawa fiye da ba "santsi" gizogizo fara da labarai shafukan ko babbar hanya kundayen da aggregators tare da manyan reference nauyi. Crawler ci gaba da yin sikanin shafukan da ɗaya ɗaya, a kan kudi da kuma daidaito na Indexing da wadannan dalilai:

• Ciki: perelinovka (ciki links tsakanin shafukan da wannan hanya), site size, daidai code, mai amfani-friendly da sauransu.
• External: jimlar reference nauyi, wadda take kaiwa zuwa shafin.

Abu na farko da bincike robot bincike a kan wani website da robots.txt. Bugu da ari hanya Indexing aka yi dangane da bayanai samu shi ne daga wannan daftarin aiki. Wannan fayil ya ƙunshi takamaiman umarnin ga "gizo-gizo" da cewa zai iya ƙara chances na page ziyara da search engines, kuma, saboda haka, ya cimma wani wuri hit site a cikin "yandex" ko Google.

Shirin analogs crawlers

Sau da yawa da Kalmar "search robot" an gauraye da fasaha, mai amfani, ko m jamiái, "tururuwa" ko "tsutsotsi". Nutsa gagarumin bambance-bambance ne kawai a kwatanta da jamiái, wasu fassarorin koma zuwa irin wannan iri mutummutumi.

Alal misali, jamiái na iya zama:

• ilimi: da shirin, wanda aka koma daga site to site, da kansa yankan shawara yadda za a ci gaba. su ne ba sosai na kowa a kan yanar-gizo;
• Mai cin gashin kanta: Wadannan jamiái taimaka mai amfani a zabi wani samfurin, search, ko ciko daga siffofin, da ake kira CD, wanda suke kadan related to cibiyar sadarwa shirye-shirye.;
• mai amfani: da shirin taimaka wa mai amfani da hulda da World Wide Web, a browser (misali, Opera, IE, Google Chrome, Firefox), manzanni (Viber, sakon waya) ko e-mail shirye-shirye (MS Outlook da Qualcomm).

"Ants" da "tsutsotsi" ne mafi kama da search engine "gizo-gizo". A farko form tsakanin wani cibiyar sadarwa da kuma consistently mu'amala kamar wannan tururuwa mallaka, "tsutsotsi" shi ne iya rubanya a cikin wasu madaidaci guda a matsayin misali crawler.

Iri-iri na search engine mutummutumi

Rarrabe tsakanin yawa iri crawlers. Dangane da manufar da shirin, su ne:

• "Mirror" - Duplicates ake lilo yanar.
• Mobile - mayar da hankali a kan mobile versions na shafukan yanar gizo.
• Quick - kafa sabon bayanai da sauri da duba latest updates.
• Reference - reference index, ƙidãya su lambobi.
• Indexers daban-daban na ciki - takamaiman shirye-shirye ga rubutu, audio, bidiyo, hotuna.
• "Kayan leken asiri" - neman shafukan da ba tukuna nuna a cikin search engine.
• "Woodpecker" - lokaci zuwa lokaci ziyarci shafukan su duba su dacewar da ya dace.
• National - lilo da Web albarkatun located a kan daya daga cikin ƙasar domains (misali, .mobi, ko .kz .ua).
• Global - index duk kasa shafukan.

Butun-butumi manyan search engines

Akwai ma wasu search engine gizo-gizo. A ka'idar, su aiki za su iya bambanta, amma a yi da shirye-shirye ne kusan m. Babban bambancin Indexing shafukan yanar gizo mutummutumi biyu manyan search engines su ne kamar haka:

• A stringency na gwaji. An yi imani da cewa ginshikai na crawler "yandex" da ɗan stricter kimomi da shafin for yarda da matsayin na World Wide Web.
• Tanadin na mutuncin da shafin. A Google crawler fihirisa dukan site (ciki har da media da abun ciki), "yandex" kuma iya duba abun ciki selectively.
• Speed gwajin sabon shafukan. Google ƙara sabon hanya a cikin search results a cikin 'yan kwanaki, a cikin hali na "by yandex" tsari na iya daukar makonni biyu ko fiye.
• A mita na sake Indexing. Crawler "yandex" duba domin updates sau biyu a mako, da kuma Google - daya kowane kwanaki 14.

Internet, ba shakka, bai iyakance don biyu search injuna. Sauran search engines da su mutummutumi wanda ya bi son Indexing sigogi. Bugu da kari, akwai da dama "gizo-gizo" da aka tsara ba manyan search albarkatun, da kuma mutum teams ko masu kulla da shafuffukan yanar gizo.

kowa tunani iri iri

Sabanin ga sanannen imani, "gizo-gizo" Ba aiwatar da bayanai. A shirin ne kawai sikanin da kuma Stores shafukan yanar gizo da kuma ci gaba da aiki daukan wani mabanbanta mutummutumi.

Har ila yau, da yawa masu amfani yi imani da cewa search engine gizo-gizo da mummunan tasiri da kuma "cutarwa" Internet. A gaskiya, wasu juyi na "gizo-gizo" iya muhimmanci obalodi cikin uwar garke. Akwai kuma wani mutum factor - da kula da shafukan yanar, wanda Ya halitta shirin, za a iya yin kuskure a cikin robot sanyi. Amma duk da haka mafi yawan data kasance shirye-shirye suna da kyau tsara da kuma fasaha gudanar, da kuma wani kunno kai matsaloli da sauri cire.

Yadda za a gudanar da Indexing

Search engine mutummutumi ne sarrafa kansa shirye-shirye, amma Indexing tsari za a iya partially sarrafawa da kula da shafukan yanar. Wannan yana taimakawa waje da ciki ingantawa na hanya. Bugu da kari, za ka iya hannu ƙara wani sabon shafin zuwa wani search engine: manyan albarkatun da musamman irin Web shafukan rajista.