Ibidem: Duplicates, renumbering, etc.

[Rick Penticoff <rpenticoff>]

Over the years I've dealt with all these issues, so here's what I know .  .  .

DUPLICATES: Pekka's original message said that he ended up with duplicates 
because he imported the same BookWhere file several times.  Thus, he ended 
up with the exact same data in two different records --- and here's the key 
point --- with two different record numbers.  So when he does a search both 
records should show up.

Technically, Pekka doesn't have duplicate records; he has duplicate 
data.  Duplicate records in the sense Mark is talking about occur when you 
make changes to an already existing record.  When you make changes, even 
something as small as substituting a semi-colon for a comma, Ibidem creates 
an updated copy of the record and appends it to the end of the data 
file.  But, if you think about it, it's not really a duplicate --- some key 
piece of data has changed.  The one thing that hasn't changed and won't 
change, no matter how many times you make revisions, is the record 
number.  In fact, you can change every single field in a record --- author, 
title, date, publisher, etc. --- and as far as Ibidem is concerned it's the 
same record you started with because the Record # has stayed the same.

When you "Maintain" a database to remove "duplicates," Ibidem searches 
through your data files for all instances of records with the same record 
number.  If it finds 5 blocks of data with the same record number it 
deletes the first four blocks and keeps the last, i.e., the most recent, 
one.  As far as I know, it doesn't look for duplicate data in any other 
field; it cares only about the record number.

I haven't tried Guido Milanese's XPL program, IBIDNO2, so I don't know how, 
or how well, it works.  If you really want, you can replace records that 
have duplicate data manually.  Find the second instance of a record with 
duplicate information.  Gather the bibliographic info for a new article or 
book.  Then just type over the data that's in the existing record with the 
info for your new article and book and presto, "duplicate" gone.  Just make 
sure you haven't ever cited the record whose info you're typing 
over.  Otherwise the new data will show up in your bibliographies, 
footnotes, and in-text citations.

Getting rid of records with duplicate information may seem obsessive, but I 
think it's not overly paranoid.  Tracking citations in a document IS tied 
to record numbers.  If you've got two separate records for Beyond Good and 
Evil (both with the exact same data) and use the first record early in a 
doc and the second record later in the doc, you (and/or NB) could get 
confused as to which one is which if you continue to cite the book.  Okay, 
okay, maybe __ I __ would get confused and the rest of you wouldn't.  As 
long as you never deleted or changed either record, I suppose it wouldn't 
matter.  Technically.  Psychologically is another matter.

RENUMBERING: Renumbering records to get rid of gaps in a database is 
possible with an XPL program written originally for NB-DOS by Tony Woozley 
and updated for NB-Win by Mary Bernard: IBIDR#.RUN.  You can download it 
from the NB Miscellany site, 
http://users.moscow.com/rpenticoff/nb/utilities/mbutils.htm  Unlike records 
with duplicate data, which _could_ cause some harm, data files with gaps 
between record numbers cause no harm at all.  If you want to renumber the 
records in your database (because a record aporia is too much to bear), 
keep in mind you'll essentially be creating a new database and that any 
documents with citations from the pre-renumbered db will have to have all 
those citations checked and potentially re-entered.  Or (the better option) 
is: before you re-number, make a copy of your db in a new directory and so 
the re-numbering on the copy.  Keep the old database as is; continue to use 
it on documents with citations from it.  Add the re-numbered db files to 
Ibidem as a new database.  Use the new db for citations in new documents.

Hope this helps.

Rick

P.S. Pekka asked what what database engine Ibidem is built on.  I don't 
know, but I doubt it's related to any of the well-known commercial desktop 
dbs (MS Access, Lotus Approach, Corel Paradox, Filemaker Pro) since Ibidem 
is a pretty simple flat-file database.