other page actions:
sorted by: recent | see :
popular
Content Tagged Xapian
Stemming prefix
Hi,
Is there a change to change the default-prefix ("Z") for
stemmed terms?
Thanks a lot!
Markus
Re: Making SORTAFTER useful in omega?
We do know that indeed. It depends on the product's category.
Perhaps you could allow to somehow script/configure extra behaviour, a
bit like all those java-solutions which can load extra
modules/classes/etc based on some (xml)configuration file.
Then you could configure that you'd want some age based posting source
to be active as soon as a flag F is supplied which is set to give items
from last week extra weight W1, items from at most half a year ago
weight W2, etc. Another useful posting source would be one that can
decide to use some value or based on the existence of some term/boolean
to give extra weight.
Best regards,
Arjen
Re: Making SORTAFTER useful in omega?
If you have a way of deciding which results want the boost, then yes.
You'll need to be able to decide that for any solution to work though.
It's not much harder to use than the Sorter. But both suffer from
requiring source code changes for use in Omega, which is a problem for
all user subclasses.
I'm not sure what the best answer is - at one point it was suggested to
rewrite Omega in a scripting language (e.g. Python) but that's quite a
job. You could also turn Omega into an API which could be wrapped with
SWIG, but that's also a fair amount of work. Both of these approaches
also mean that you incur the overhead of a trip from C++ to Python and
back for each operation, which can add up when you do it for a million
documents.
Allowing plugins to Omega would mean you'd not need to recompile the
whole of Omega, just the plugin, but I'm not sure that really makes it
significantly less hassle.
Cheers,
Olly
Re: Problem with stemming and prefixes.
You want add_boolean_prefix() here. add_prefix() is for "free text"
like fields.
Cheers,
Olly
Re: xapian-compact and "Bad line 1 in stubdatabase file"
Are you perhaps trying to open "index.compact/postlist.baseA" as a
database (rather than "index.compact")?
That's OK - after compaction, the "A" revision is for an empty table,
the "B" revision is for the table with all the data added.
If it's not the database path, please try 1.0.8.
Cheers,
Olly
Re: xapian-compact and "Bad line 1 in stubdatabase file"
I'm pretty sure I am using the directory.
[...]
Okay, I will do that and report what happens. Thanks!
Best regards,
Adam
Problem with stemming and prefixes.
Hi there,
I am encountering problems with stemming and prefixes.
I want to filter my queries with prefixes using add_term and add_prefix (as
I've seen it's the best solution).
So, here is the situation : my database contains movie titles, movie
categories and production years.
I do not filter the titles, but I filter the categories and the years, using
:
$doc->add_term("XYEAR".mysql_result(/* request to the year's field */));
$doc->add_term("XCAT".strtolower(mysql_result(/* request to the category's
field */)));
Note that the strtolower() is not the source of the problem and it is
mandatory.
Then, I set the add_term (I stop with the "year" case because it works) :
$queryparser->add_prefix("cat", "XCAT");
Then I call my request through Firefox :
http://coolurl/search.php?myssearch=cat:Thriller
That works. It returns all the thriller movies.
But this :
http://coolurl/search.php?myssearch=cat:thriller
Do not work.
Indeed, a "print $query->get_description();" tells me that the queries are
respectively :
X
Re: Problem with stemming and prefixes.
As you know that it is a cat search, from your prefix "cat", the
simplest solution is to upper case the first letter using the function
ucfirst.
xapian-compact and "Bad line 1 in stub databasefile"
I have a xapian index on a test-machine that has been used quite a bit.
The size of the index on disk is 2.4GB, so I thought I'd run
xapian-compact on it, to make it more likely to fit in RAM.
It went like this:
$ time xapian-compact index index.compact
postlist: Reduced by 50.0176% 387080K (773888K -> 386808K)
record: Reduced by 40.8424% 5120K (12536K -> 7416K)
termlist: Reduced by 42.869% 62064K (144776K -> 82712K)
position: Reduced by 44.1926% 651760K (1474816K -> 823056K)
value: Reduced by 47.2156% 6376K (13504K -> 7128K)
spelling: Size unchanged (0K)
synonym: Size unchanged (0K)
real 7m18.317s
user 0m46.211s
sys 0m10.477s
$
A nice reduction to 1.3GB.
But when I try to use the new index in place of the non-compact one, it
doesn't work, the application says:
Exception: Bad line 1 in stub database file `index.compact/postlist.baseA' at
/usr/lib/perl5/Search/Xapian/Database.pm line 48
If I look at the file-size:
-rw-rw-r-- 1 adsj staff 13 2008-10-08 20:01 index.compact/
Re: mistake inhttp://xapian.org/docs/valueranges.html
It should indeed - now fixed in SVN. Thanks.
And this explicit empty destructor isn't needed, so I've removed it.
Cheers,
Olly
mistake in http://xapian.org/docs/valueranges.html
I think there is a mistake in http://xapian.org/docs/valueranges.html,
SECTION "Custom subclasses", source followed:
struct AuthorValueRangeProcessor : public Xapian::ValueRangeProcessor {
AuthorValueRangeProcessor() {}
~AuthorValueRangeProcessor() {}
Xapian::valueno operator()(std::string &begin, std::string &end) {
if (begin.substr(0, 7) != "author:")
return Xapian::BAD_VALUENO;
begin.erase(0, 7);
begin = Xapian::Unicode::tolower(term);
end = Xapian::Unicode::tolower(term);
return 4;
}
};
term is the parameter of tolower in it's definition:
begin = Xapian::Unicode::tolower(term);
end = Xapian::Unicode::tolower(term);
shoule be :
begin = Xapian::Unicode::tolower(begin);
end = Xapian::Unicode::tolower(end);
Re: trouble on getting source
Please read the website before asking further questions. You're wasting
a considerable amount of the developers time, which could be better
spent improving Xapian.
The answer in this case is to be found on the "bleeding edge" page,
linked to from the front page of the website. Or, you could have
searched for "svn" in the search box on the front page of the website,
which would have returned that page at the top of the result list. Or,
you could have searched on Google for "xapian subversion", which would
have returned a mailing list message pointing to the web interface to svn.
I won't answer any further questions which are this easy to find the
answers to.
Re: trouble on getting source
If you just want the source for xapian, it's the first link on
http://xapian.org/download.php
If you're wanting the source for the Debian package, you need to use the
"apt-get source" command: probably "apt-get source libxapian15"
(libxapian15 is the most recent Xapian library package). This is a
basic Debian usage question, rather than a Xapian question, though -
this mailing list isn't an appropriate place to ask for help with using apt.
Re: trouble on getting source
Is there any CVS or SVN of xapian?
2008/10/7 Richard Boulton lemurconsulting.com>
Re: trouble on getting source
I solved it by modifing source.list, change stable to etch :
deb http://www.xapian.org/debian etch main
deb-src http://www.xapian.org/debian etch main
2008/10/7 Olly Betts survex.com>
Page 1 | Next >>
