The spiders attack...
Ok, serious....
What are those Google Spiders?? There is Spiders everywere. Just look in "Who's online" and see by yourself. I'm so curious... |
Theres another thread on this furthur down the list..
anyway, they're search engine bots which save pages into the Google/Lycos etc database so they can link to it if people search for keywords on that page... Or, as Death aptly put it: "Spiders are not quite "Bots" as such. Spiders do exactly what spiders do in real life. They crawl the web, downloading things as they go. Spiders often download and save entire websites just to provide faster searches and word matches. They only do it on popular sites though, so this site must be quite popular. Another feature of a spider is to gather information such as statistics, and hits. This is to figure out whether the website the spider is visiting is popular or unpopular. If it's popular, a second spider will be called in, and the two spiders will systematically download the website, and add it to the search info database. The reason spider bots are assosciated with websearches, is because a spider bot gets sent around the web on most searches, looking for content that the database does not yet have. The spiders' first role though, is to check the database to see if any matches can be found. This all happens in a matter of miliseconds, and gets faster and faster as time passes. A searchbot, is quite similar to a spider, only that it just seeks out content matches, then generates a link using php. The searchbots' role is really to take snippets of relevant data, that can be processed through php to appear on a results page. The spidered database pretty much "hints" the searchbot on where to look, and what to look for. For instance, if you searched for the word "Oddworld", the searchbot would ask the spiderbot to check the database for any websites that are listed to contain that word, then the spiderbot would tell the searchbot which ones contain that word, where they contain it, and what the URL is - once it does that, it puts those links onto the results page ready to go into php - then it searches the net accompanied by a spider, looking for content matches. Once those are found, links are created, and the list of results are put in order of best match to worst match. Again, that all happens in a matter of miliseconds. The difference between a spider and a searchbot, is that the spider deals with content, and the searchbot deals with searches. Originally, there were only searchbots, and searches used to take ages, but now you can get to the most popular site in much less time." |
Ohh, thanks for the explenation :) you really cleared that up...
Thanks again :D |
You people have noticed that the most users online is 140, right? That's bullshit.
We have like... 14. |
That's because guests and spiders are counted towards the most people online statistic :p It was like 50, before the spiders arrived that one time, not so long ago.
Alcar... |
Ya know on a more silly note the spidar and searchbot concept reminds me of science fiction like the Matrics when you have those sentinal droids. But yeah thats not relevant at all.
Paramiteabe... :fuzblink: |
:
|
Yes you are.
so these things arnt a threat then? |
|
Aw, poor Jeeves, he's all alone:p
|
:
|
:
|
Who's Online is really a tool for the management to use to keep track of where members are. As such, things like spider recognition are turned on so we know exactly what's happening.
Alcar... |
I use 'Who's Online' when I'm seeing if the cool, kids are on. Also if I sent a PM and I'm getting impatient for a reply. I'll just go to who's online and be like, "What the hell are they doing?! Oh...PMing...good boy! :D" Of course, I don't really talk to myself in such a psycho-maniac manner...really. :dodgy: It must be late, I'm acting weirder than usual.
*looks at clock* Oy...11:16pm -oddguy |
You get psycho at 11:00? Pffft :rolleyes: I can go without being psycho until 3 am (that's usually when the psycho parties are).
I'm constantly visiting the "Who's Online" list for the same reason as oddguy, aswell as seeing who's replying to what so I can rush over there and be the next to reply. :p |
I used to know of someone who used to like "sit" on the who's online page constantly just waiting for stuff. I remember once they were there for like half a day ... talk about obsessive!!!
Those spiders make our site seem really popular ... we'd never get 140 people online at one time. Hell, many larger sites wouldn't even get that. Abe Babe... |
Perhaps the demand for Oddworld 4 information has triggered the numerous search engines to crawl all Oddworld related websites deeper, thus the massive amount of spiders. It's a good thing they only send out the massive amounts of spiders occassionally, to update their indexes.
Otherwise, I'd fear for our bandwidth. Alcar... |
There is a reason why spiders have to keep coming back here. Because the content changes so often, the website must be re-spidered all the time just to allow for content matches to be provided for new content in posts. This happens on all forums that are registered with search engines.
The more content that is spidered, the more people will generate matches to this site. Thus, spiders are good for generating traffic. They can be a bit of a problem with bandwidth though, but usually they suck up no more than a single guest. |
Here's something new....A spider registering?
|
Probably just saving the register page in cache...err something like that.
-oddguy |
It is merely the spider finding it's way into the first registration step. Unless Google has suddenly come up with a super intelligent AI system capable of deciphering human text and numbers, it won't actually register :p
Alcar... |
:
:
Registerbots can be set up with their own SMPT mail engine so they generate their own email address, and sign up with that - any data that gets sent to them from forum registrations gets analyzed accordingly. |
What I mean, is the image verification. Not the username, email, password fields. I'm sure there are such things around, but the degree of difficultly is extreme (at least, with our common civilian machines :p).
Alcar... |
|
I found one too!
MSNBot Spider :) But it's really funny. Over the last 3 weeks, every time I've looked in "Who's Online", I've seen search engine spiders there. It's weird. - DH |
Hehe. I updated the list of Spiders yesterday evening. We went from seven Spiders, to at least 100 :p But there will still be Spiders out there that we haven't got in our lists.
Alcar... |
:
-oddguy :cool: |