NFS weirdness...

[Dave Berry]

I currently work on a solr/lucene config with 1 master/3 slaves
updated via solr's snappuller(fancy rsync) because of the NFs
problems.

On Thu, Apr 16, 2009 at 4:17 PM, ref <tbs-Gb/NUjX2UK8 at public.gmane.org> wrote:
> On Thu, 2009-04-16 at 15:50 -0400, Dave Berry wrote:
>> What version of Lucene are you using?   Lucene is not recommended to
>> run over NFS due to locking problems on the NFS and index side.  I
>> realize that this may not be your ideal situation but Lucene can use a
>> master-slave^N setup where you have 1 writer and use rsync to keep the
>> slaves up to date.
>>
>> Links about lucene/NFS-The first link is old(2005) but the second
>> references NFS and Lucene 2.0
>>
>> http://www.opensubscriber.com/message/lucene-user-sSm70nXyYxXWgOoZo4XcuR2eb7JE58TQ@public.gmane.org/800819.html
>>
>> http://www.gossamer-threads.com/lists/lucene/java-user/60163
>>
>>
>> -dave
>
>
> Thanks Dave, this is what I was afraid of. Looks like we'll have to use
> solr, and retro fit it. ah well, it keeps us busy :)
>
>
>>
>>
>> On Thu, Apr 16, 2009 at 3:32 PM, Jerry Feldman <gaf-mNDKBlG2WHs at public.gmane.org> wrote:
>> > How are the Lucene Index directories being exported, and what parameters are
>> > you using to mount. This looks to me like more of a networking issue.
>> > Make sure your switch/router is solid. I did a quick google search of .nfs
>> > files since I wanted to present a better answer.
>> > http://opensolaris.org/jive/thread.jspa?threadID=84407&tstart=0
>> > ".nfs files are created when a file is deleted that is still open. "
>> >
>> > The above URL refers to Solaris, and is not 100% relevant. You can also use
>> > the lsof command to see if it refers to any open file.
>> >
>> > My thoughts are (1. network flakyness, 2. the a program dies before properly
>> > closing the nfs file.
>> >
>> > Since it appears that the Lucene Indexes are not modified, you probably
>> > should export those directories read-only if you don't know.
>> >
>> > On 04/16/2009 01:14 PM, ref wrote:
>> >>
>> >> Hi,
>> >>
>> >> I am using NFS in a production environment to allow multiple front end
>> >> servers to access a data server containing Lucene indexes.
>> >> Recently, the NFS connections have been acting weird and the index files
>> >> got badly corrupted. While investigating, I saw a load of zero byte
>> >> files in the NFS mounted directory ...  Is this normal ? what are these
>> >> files ? can I bit-bucket them ? are they important ? I have not seen
>> >> them before/become aware of them till now ...
>> >>
>> >>
>> >> -rw-rw-r--  1 production production     0 Apr 15
>> >> 14:09 .nfs016007720001e150
>> >> -rw-rw-r--  1 production production     0 Apr 15
>> >> 16:10 .nfs016007ea0001e270
>> >> -rw-rw-r--  1 production production     0 Apr 16
>> >> 06:11 .nfs016008e20001e742
>> >> -rw-rw-r--  1 production production     0 Apr 16
>> >> 04:11 .nfs016009a80001e6bc
>> >> -rw-rw-r--  1 production production     0 Apr 15
>> >> 16:10 .nfs01600a270001e246
>> >> -rw-rw-r--  1 production production     0 Apr 16
>> >> 10:09 .nfs01600a560001e85e
>> >> -rw-rw-r--  1 production production     0 Apr 15
>> >> 14:09 .nfs01600a790001e14f
>> >> -rw-rw-r--  1 production production     0 Apr 15
>> >> 14:09 .nfs01600a860001e151
>> >> -rw-rw-r--  1 production production     0 Apr 15
>> >>
>> >>
>> >> any help most welcome (rebuilding lucene indexes is a time consuming
>> >> process and I'd rather not have to do it regularly!)
>> >>
>> >> thanks,
>> >>
>> >> Richard
>> >>
>> >
>> >
>> > --
>> > Jerry Feldman <gaf-mNDKBlG2WHs at public.gmane.org>
>> > Boston Linux and Unix
>> > PGP key id: 537C5846
>> > PGP Key fingerprint: 3D1B 8377 A3C0 A5F2 ECBB  CA3B 4607 4319 537C 5846
>> >
>> >
>> >
>> > _______________________________________________
>> > Discuss mailing list
>> > Discuss-mNDKBlG2WHs at public.gmane.org
>> > http://lists.blu.org/mailman/listinfo/discuss
>> >
>> >
>>
>> _______________________________________________
>> Discuss mailing list
>> Discuss-mNDKBlG2WHs at public.gmane.org
>> http://lists.blu.org/mailman/listinfo/discuss
>
>
>