Page 1 of 1

new search web query

Posted: Sat Jun 26, 2010 9:51 pm
by florachan
Great job eric,

while I am very impressed with the new search web query function. I am wondering if the following function can be implemented

Local Host Last Access Time Search Text

At the moment it only shows the Searched Text, however it doesn't tell you where or which portal that the search query is from. Not until I correlate the search time and the domain access time, that I know that search query is from google.

I am wondering it is possible to output

Local Host, Last Access Time, Search Text, "Search Domain"

this will provide information where that search query from, eg from YahooSearch.


also it seems like excluding IP from search query can only exclude one IP only. When I put the second IP in, it doesn;t save.

Re: new search web query

Posted: Sun Jun 27, 2010 9:30 am
by BikeMike
florachan wrote: also it seems like excluding IP from search query can only exclude one IP only. When I put the second IP in, it doesn;t save.
Yes I noticed this too, actually the additional IPs are being saved (and therefore not monitored), but only the first is displayed.

I agree with your suggestion too if it's not too hard. Also I'd like to know how this works, is there a list of sites it recognizes as search engines, or some other method?

Re: new search web query

Posted: Sun Jun 27, 2010 9:43 pm
by florachan
Also I'd like to know how this works, is there a list of sites it recognizes as search engines, or some other method?
rather than just limit it to list of site...

I thought just displaying the IP address of where the Search query is from, then do a reverse whois lookup just like web monitor does.

Re: new search web query

Posted: Tue Jun 29, 2010 6:40 am
by BikeMike
I've noticed perhaps another bug with the "Recent Web Searches" log. Sometimes when a long search query is entered, it shows up in the log many times over a few seconds, each one with an extra letter added in the middle. So it starts off with say the first 2 & last 2 words mashed together, then extra letters added to the first words so it's still readable...
Image

Re: new search web query

Posted: Wed Jun 30, 2010 8:53 am
by Eric
BikeMike try version 1.3.2, I think this issue has been addressed.

What's happening is that the search engine is starting a query before the user is done typing. So all those queries are real... it's just that they aren't what the user inteded to look for.

I've added a fix that will discard requests made within 1 second of each other, but sometimes I still see one or two searches that are a result of the user querying one thing (though this is much better than the situation where 50+ searches are being saved that can occur as you show in your screenshot).

There's a trade-off between potentially discarding searches you want because they were made too close together and discarding searches caused by this query-as-you-type issue.

The implementation is an iptables module, and there's a (long) hard-coded list of search engines in their. It's a giant if/else if statement.

Also, the bug that BikeMike refers to, which causes only half the IPs to include/exclude from monitoring to actually show up after saving has been fixed.

I may get around to adding another column for the query the search came from as BikeMike suggests (I like this idea), but I'm not sure when I'll get to it. No promises on that.

The implementation is a huge if/else if clause inside an iptables module. It's essentially a hard-coded list of search engines. It's really kind of ugly, but it works.

Re: new search web query

Posted: Thu Jul 01, 2010 8:40 am
by BikeMike
Thanks for the info and fixes Eric.

I can now enter multiple IP's to exclude and it works fine, also the search logging seems clear of the multiple entries.

One problem that I've just noticed, so don't know if it's new or was already there, is the Clear History button doesn't work. I've tried it several times, tried save settings, navigating away and back to the page, but the history remains.

Re: new search web query

Posted: Thu Jul 01, 2010 8:13 pm
by florachan
thanks eric... i tried the ignored the users now and it is working now and showing it


really looking forward to the following feature, which is nice to have.

Local Host, Last Access Time, Search Text, "Search Domain"