#1 2010-08-07 04:48:37

b-man
New member
Registered: 2010-08-07
Posts: 1

GCStar not importing Alexandria data.

Hello.

I have just installed GCStar, as i started needing a software package that is both cross-platform and does more than book collections (both of which Alexandria does not do).

I'm using Kubuntu Lucid 10.04, and installed GCStar 1.6.0 out of the PPA's (after some dancing around, as by default apt-get only installs 1.5 for some reason; i had to go and manually install 1.6 by hand out of the devel-ppa).

I tried importing my alexandria data. After starting GCStar, i get the "select a collection type" dialog box, and go straight down to the Import>Alexandria. After this, the programme just goes grey, with the progress bar at the bottom not doing anything. I tried leaving it for a couple of hours, but nothing happened.

Also, i don't think GCStar itself hangs, as when i click on the X on the window title bar, the programme closes straight way (if truly were hung, with would have needed a ctrl+alt+esc; at least they do in my experience).

This problem is consistently reproducible, and i've had it before with previous versions.

I have a considerable collection in Alexandria, so adding individual entries by hand is not really an option.

Advice / suggestions?
Cheers!
b

Offline

 

#2 2010-08-07 05:18:20

zombiepig
Moderator
Registered: 2007-08-30
Posts: 331

Re: GCStar not importing Alexandria data.

Ok, I can reproduce this. The solution/workaround is to create a new blank collection first - so make a books collection, then File->Import.

I'll make a bug report about this one.

Offline

 

#3 2010-08-07 05:21:44

zombiepig
Moderator
Registered: 2007-08-30
Posts: 331

Re: GCStar not importing Alexandria data.

Offline

 

#4 2018-05-20 09:37:03

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

Hello,
Sorry to dig up this thread after so long time but I face another problem after applying the workaround :

Code:

Gtk-Message **: Failed to load module "canberra-gtk-module" at /usr/lib/x86_64-linux-gnu/perl5/5.26/Gtk2.pm line 126.
*** unhandled exception in callback:
***   Wide character at /usr/lib/x86_64-linux-gnu/perl/5.26/Encode.pm line 212, <BOOK> line 22.
***  ignoring at /usr/bin/gcstar line 328, <BOOK> line 22.

This seems to be a problem of string encoding/decoding.

I am on debian Buster. I would like succeed to import an Alexandria multiple library database, thanks for your help.

G.

Offline

 

#5 2018-05-20 16:56:50

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

The Perl functions of the Alexandria importer has not changed in the last 10 years so something must have changed in the files you're trying to import.

To understand what's happening it could be great to have a file sample able to reproduce the error.

If necessary, I could post a modified version of the GCImportAlexandria.pm with some trace messages to help identity the corrupted items.

Offline

 

#6 2018-05-21 21:09:18

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

You're right, it comes from an Alexandria yaml file which describe a book. I found one which produces the crash.
I guess it contains a wrong utf-8 multi-byte character on the second word 'Glénat'. See attached crashing_file_9782344001615.yaml.
When I changed the 'é' to 'e' it worked fine. When I put 'é' again it was still working !
So I used 'bless' program to show hex flow of both working files with 'é' inside, see attached compare.png. The crashing file has one char more than the working one.

I remember there was an update of Alexandria not so far ago when I needed to reword a lot of non ascii chars of my books, there was maybe something wrong...

I should probably open a bug on Alexandria side. Now I know how to import my books but it is painful to locate crashing files. I used dichotomy. And I know there is one more at least in another library with more than 200 books. Maybe you can add a small part of code to be able to pass over wrong charset conversion.

Note that yaml files was named : 9782344001615.yaml


Attachments:
Attachment Icon crash.zip, Size: 173,257 bytes, Downloads: 32

Offline

 

#7 2018-05-22 06:47:33

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

Sorry but there seems to be a problem with the forum and I can't download the attached archive to investigate further.

Offline

 

#8 2018-05-22 08:17:50

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

You may try to replace your current GCImport/GCImportAlexandria.pm by the attached file. It should trace the Alexandria files as they are parsed to help identify the bad ones.


Attachments:
Attachment Icon GCImportAlexandria.pm, Size: 4,742 bytes, Downloads: 32

Offline

 

#9 2018-06-09 16:10:28

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

Hello,

Thank you so much, I was able to find the only second book which didn't import !
There seems to be a problem with Author and Tag fields for which I had one on every book in Alexandria, there are a lot missing in Gcstar.

I also see two optimizations which can be done in the import module :
1. Fill the Read and Publication Date fields from Alexandria one.
2. Import strings as utf-8 so I wouldn't have to change all special chars.
That would be very nice !

Thank you.
Guillaume

Last edited by ghpom (2018-06-09 16:42:34)

Offline

 

#10 2018-06-09 18:20:39

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

I'm glad you could find what was wrong.

To be able to see if the import program can be improved, I need to have some Alexandria files. This forum seems to have problems with ZIP files, you could maybe try to upload again the crash.zip file by renaming it to something like crash_zip.txt.

Offline

 

#11 2018-06-10 07:28:03

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

Hello Kerenoc,

I've think about some more optimizations :
3. Concerning the tag field of Alexandria, I used it to mark the book kind (Novel, comic... which is Roman, BD in french). Maybe it can be imported in the Kind specific field of GCstar directly. It can be an import options.
4. Secondly, there is a boolean field Owned in Alexandria, maybe it can be imported as importing date in Acquisition field of GCstar if true.
5. Maybe the borrower name from Alexandria can be automatically created and assigned in loan tab instead of Unknown.

I've prepared a small Alexandria folder with a few books inside, with some different importing results regarding previous points (see, attached) :
1 : all books
2 : 9782731614268
3 : all books
4 : all books
5 : 9782757823040
There are a few more files in the folder :
- the two files which doesn't import : '9782266003346.yaml' and '9782344001615.yaml'.
- one *.cover.orig : Alexandria sometimes create a backup of the book picture. Just for information so GCstar can ignore it properly.
- one valid book's yaml file for which the name isn't an ISBN : '-3711881059468292551.yaml' and '1837862428990976495.yaml'. This is due to the book doesn't have an ISBN because too old or not a book that can be sold or something like that.

For the 5th point, it is maybe complicated to implement and I have a few books concerned so I can handle it by hand. For others it would be very helpful to me if you succeed to implement them because I have 438 books in my collection (yes I told 200 previously but it seems I didn't looked at this since a while smile), thank you by advance to take time to look at this.
!


Attachments:
Attachment Icon .alexandria-to improve importer.zip, Size: 48,003 bytes, Downloads: 25

Offline

 

#12 2018-06-11 18:23:07

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

Hello ghom

Thanks for providing the Alexandria files. I was able to improve the GCImportAlexandria.pm file.

Feature 1: fields imported
Feature 2: UTF8 is use as the default encoding and the call to "encode" that doesn't seem to support Unicode combining characters is avoided
Feature 3: the tags are duplicated in the "genre" field
Feature 4: the acquisition date is set to the date of importation as it doesn't seem to be available in Alexandria
Feature 5: in GCstar, you need to create the borrowers before importing then everything should be fine.

You can test the mods using the link above or you can update all Perl files using an archive from the Gitlab repository to benefit from the latest changes.

I installed Alexandria on a Raspberry Pi to check it : it seems that GCstar is more powerful and I didn't find something that is not supported in GCstar. I understand why one should migrate!

By the way, I'm French : thanks for reading French books or for using English for the benefit of the non-French speaking GCstar users!

Offline

 

#13 2018-06-11 21:45:54

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

Hello,

Thank you ! Everything seems fine except the Read field which is inverted. Can you have a quick look at this ?
I succeeded importing in a good way including loans and strings in utf-8.

Just for my information, did you put the Tag copy to Genre field under a checkbox option ? Because I'm not sure that every body wants this behavior. However, thank you so much, you saved me a painful work !

When I chosen to use Alexandria (around 2010), it was still packaged in both ubuntu and debian. I tried GCstar but I found the Alexandria UI more user friendly and I didn't needed powerful features. I didn't seen that it was quite not supported by a community anymore. Today the developer that support it only keeps environment and dependencies working (ruby). Then I use GCstar for two other databases.

You're welcome, I'm used to write english at work, so it isn't a big deal to me...
Again, thank you !
G.

Offline

 

#14 2018-06-13 12:02:26

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

Value of Read tag fixed in latest version on Gitlab : thank you for noticing.

Offline

 

#15 2018-06-13 22:01:14

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

I downloaded the file with your last patch from Gitlab and it wasn't working. I had to invert values between the two below lines to get it work :

        $book{read} = 1 if $book{read} eq 'true';
        $book{read} = 0 if $book{read} eq 'false';

So it was working with :

        $book{read} = 0 if $book{read} eq 'true';
        $book{read} = 1 if $book{read} eq 'false';

This seems strange... My GCStar version is 1.7.1.

Is there another inverting code elsewhere ?

Offline

 

#16 2018-06-14 22:46:42

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

Strange indeed. In my environment, when I check "Read" in Alexandria, the "redd" parameter is set to true in the YAML file.

After importation in GCstar, the parameter "read" is set to 1 in the GCS file.

If you're still using 1.7.1 I suggest that you install the latest unofficial version from the Gitlab repository. It still pretend to be 1.7.1 (no news from the original author to be allowed to create a new version) but it provides more than one year of changes.

Offline

 

#17 2018-06-16 10:59:23

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

In my alexandria folder the field redd is false when I own the book. It means "to be read" and the field "redd_when" means ''before date".

Ok, I will have a look on how to install the latest version.

Offline

 

#18 2018-06-17 08:14:07

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

In the version of Alexandria I used to test (from Github), the Read and Own field are independant and can be set separately. In the other hand, the Want field  is correlated with the Own field.

Offline

 

#19 2018-06-17 08:24:40

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

Sorry my bad, I mixed them...
I finally succeeded installing Alexandria so I double checked : Redd field is named "Read it ?". So it definitively means "to be read" and the field "redd_when" means ''before date".
You should invert the import logic.

Offline

 

#20 2018-06-18 07:04:05

kerenoc
Member
Registered: 2016-03-19
Posts: 210
Website

Re: GCStar not importing Alexandria data.

I checked the Alexandria source code, in the import section from CSV : "redd" doesn't mean "to read"

Code:

      if row[@read_count]
        count = row[@read_count].to_i
        book.redd = true if count > 0
      end
      if row[@date_read]
        date = Date.strptime(str, '%d/%m/%y') # e.g. "14/01/10" => 2010-01-14
        book.redd_when = date
        book.redd = true
      end
      if row[@mainshelf]
        if row[@mainshelf] == 'read'
          book.redd = true
        elsif row[@mainshelf] == 'to-read'
          book.redd = false
          book.tags = ['to read']
        elsif row[@mainshelf] == 'currently-reading'
          book.redd = false
          book.tags = ['currently reading']
        end
      end

I think the Alexandria author may not be a native English speaker and that the "Want" field should have be named "Wanted" in the interface so that Read, Wanted and Owned would have be all past participles (Read being prononced /red/).

If you want your own behaviour and semantics, you can patch the file GCImportAlexandria.pm accordingly.

Otherwise, you can use the "Find and replace" function of GCstar to change the values in you collection.

Regards

Offline

 

#21 2018-06-19 19:49:00

ghpom
New member
Registered: 2018-05-20
Posts: 9

Re: GCStar not importing Alexandria data.

OK, got it. It's me who am using this flag the inverted way. However, I already patched the GCImportAlexandria file so it fitted my need.

Again, thank you so much for support !
G.

Offline

 



Should you have a problem using GCstar, you can open a bug report or request some support on GCstar forums.