• Index
  •  » Support
  •  » Help with XML import from Data Crow?

#1 2021-11-07 19:38:55

nobody
Member
Registered: 2021-11-07
Posts: 17

Help with XML import from Data Crow?

hello,

i have movies inventoried in a data crow 4.2.1 database and i am interested in trying out GCstar. i have exported the data crow database to xml and tried using the GCstar Data Crow (.xml) import option however nothing appears to happen after clicking "Import".

i have tried both the "Create a new collection" option and the "Add to current collection" option in the GCstar import window.

any help is greatly appreciated.

here is what the data crow xml file looks like:

Code:

<?xml version="1.0" encoding="UTF-8" ?>
<data-crow-objects xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Data Crow.xsd">
    <movie>
        <item>Movies</item>
        <actors>
        </actors>
<actors-list></actors-list>
        <aspect-ratio></aspect-ratio>
        <audio-bitrate></audio-bitrate>
        <audio-channels></audio-channels>
        <audio-codec></audio-codec>
        <audio-languages>
        </audio-languages>
<audio-languages-list></audio-languages-list>
        <audio-sample-rate></audio-sample-rate>
        <available>true</available>
        <certification></certification>
        <color></color>
        <container>
            <container>
                <name>Digital Video Non-Fiction</name>
            </container>
        </container>
<container-list>Digital Video Non-Fiction</container-list>
        <countries>
        </countries>
<countries-list></countries-list>
        <created>2019-06-18</created>
        <days></days>
        <days-loaned></days-loaned>
        <description>Description of the movie</description>
        <directors>
        </directors>
<directors-list></directors-list>
        <due-date></due-date>
        <ean></ean>
        <episode-nr></episode-nr>
        <episode-name></episode-name>
        <filehash></filehash>
        <filehash-type></filehash-type>
        <filename></filename>
        <filesize></filesize>
        <fps></fps>
        <frames></frames>
        <genres>
        </genres>
<genres-list></genres-list>
        <height></height>
        <id>0018e998-ab90-4334-968f-ed31de0ccc00</id>
        <item>Movies</item>
        <item-value></item-value>
        <label>Movie Title</label>
        <languages>
        </languages>
<languages-list></languages-list>
        <lend-by></lend-by>
        <loan-end-date></loan-end-date>
        <loan-start-date></loan-start-date>
        <loan-status>Available</loan-status>
        <modified>2019-06-18</modified>
        <picture-back></picture-back>
        <picture-cd></picture-cd>
        <picture-front>file:///C:\Path\To\Data Crow_images\0018e998-ab90-4334-968f-ed31de0ccc00_PictureFront.jpg</picture-front>
        <playlength></playlength>
        <rating></rating>
        <season></season>
        <series></series>
        <service></service>
        <service-url></service-url>
        <state></state>
        <storage-medium>DVD</storage-medium>
        <subtitle-languages>
        </subtitle-languages>
<subtitle-languages-list></subtitle-languages-list>
        <tags>
        </tags>
<tags-list></tags-list>
        <title>Movie Title</title>
        <title-local></title-local>
        <user-long-text-field></user-long-text-field>
        <user-numeric-field-1></user-numeric-field-1>
        <user-numeric-field-2></user-numeric-field-2>
        <user-short-text-field-1></user-short-text-field-1>
        <user-short-text-field-2></user-short-text-field-2>
        <video-bitrate></video-bitrate>
        <video-codec></video-codec>
        <webpage></webpage>
        <width></width>
        <year>2004</year>
    </movie>
    <movie>

Offline

 

#2 2021-11-10 22:53:04

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

Looking at the GCstar source code (https://gitlab.com/Kerenoc/GCstar/-/blo … row.pm#L95) it appears that currently it can only import books collections from DataCrow. It should be simple enough to test if the model ($self->getModelName) is GCbooks or GCfilm and to add the conversion of the movie XML data structure to the internal GCstar structure.

Offline

 

#3 2021-11-10 23:16:44

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

happy to help test in any way possible.

Offline

 

#4 2021-11-17 09:31:16

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

I added the support of movies for the DataCrow import. Some basic fields are currently imported (code), more can be supported if necessary.

Offline

 

#5 2021-11-18 18:42:56

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

thank you very much - will test it out

Offline

 

#6 2021-11-30 05:03:09

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

finally got a chance to test the import and it works now, thank you. there are a few things i'm trying to figure out if you have any additional thoughts/tweaks?

1. images seem to import correctly on windows but don't seem to be importing on a linux test machine. looking at the getPictureName function in the pm file but not entirely sure what it's doing. currently i have the Data Crow xml file in a directory, then all the images in a 'Data Crow_Images' subdirectory in the same directory. (will try getting rid of the space)

2. have gotten a few additional fields to work correctly by adding them to GCImportDataCrow.pm but can't get the following to work:

a. $item->{added}     = $movie->{created};

nothing imports, entries in the data crow xml look like: <created>2019-06-18</created>

b. $item->{tag}     = $movie->{'tags-list'}; (have also tried $item->{tags})

again nothing imports, entries in data crow xml:

<tags>
            <tag>
                <name>This Is A Tag</name>
            </tag>
        </tags>
<tags-list>This Is A Tag</tags-list>


3. lastly a minor thing: some empty fields in the data crow xml, for example genre, will get a genre entry in gcstar upon import like 'HASH(0x113fdc58)'.

thank you again so much

Last edited by nobody (2021-11-30 05:04:47)

Offline

 

#7 2021-12-01 09:01:47

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

Thanks for testing and contributing to improve GCstar.

Some possible fixes to the problems are now on the Gitlab repo : please tell us if it improves the situation.

Offline

 

#8 2021-12-01 23:22:25

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

wow, thank you! import now works perfectly for the add date and tags. also the HASH entries are gone/corrected.

one last (i think last!) small question. is there an easy way to amend $item->{genre}    = $movie->{'genres-list'}; to first import the genres-list values and then append any containers-list values afterwards?

if it is helpful to anyone else, here is the fix i used for getting images to import correctly in linux:

was able to get the images to import correctly in linux by removing spaces from the directory names and then doing a find and replace in the data crow xml file. (upon review, just noticed further user error before. i moved the directory after running the data crow export so it had the wrong windows style path in there anyways)

file:///E:\Data Crow\Data Crow_images\0018e998-ab90-4334-968f-ed31de0ccc00_PictureFront.jpg

with

/mnt/shared/DataCrow/DataCrow_images/0018e998-ab90-4334-968f-ed31de0ccc00_PictureFront.jpg

Last edited by nobody (2021-12-02 05:38:54)

Offline

 

#9 2021-12-03 08:29:18

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

I tested migration from DataCrow on Linux to GCstar on Linux, starting from a user directory with space in the name : the images was correctly imported. It means that the Data Crow export format does not support migration between OS as it uses an absolute path name for images. You may try to use the "backup/restore" function of Data Crow to test if it has the same problem.

As for containers, I may add them to the tags when importing but I don't fully understand the meaning of this field. In GCstar, the "location" field is used to identify the physical location of a collection item (I use it to record the room and shelves for my book collection), it may be more appropriate for containers than the "tags" field.

Offline

 

#10 2021-12-04 06:08:52

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

pardon my confusion about the space in the name, it was just one idea, but i think mostly irrelevant. what i was trying to say in the last post was that i was able to get images to import a data crow profile exported in windows into gcstar on linux by updating the path name in the data crow export .xml file.

for example file:///E:\Data Crow\Data Crow_images\0018e998-ab90-4334-968f-ed31de0ccc00_PictureFront.jpg becomes /mnt/shared/DataCrow/DataCrow_images/0018e998-ab90-4334-968f-ed31de0ccc00_PictureFront.jpg

by finding and replacing all instances of file:///E:\Data Crow\Data Crow_images\ with /mnt/shared/DataCrow/DataCrow_images/

---

in regards to containers: i think you are right to associate them with 'location', i think that's their intended use. i used them in a somewhat different manner to create sub-collections - 'Digital Fiction', 'Digital Non-Fiction', 'Non-Digital Fiction', 'Non-Digital Non-Fiction' (non-digital for physical items).

for this specific case i was just wondering if there was a fairly simple tweak i could make to the .pm file to get the import to append the containers-list items to the genre field, after the genres-list items.

thanks again

Last edited by nobody (2021-12-04 06:10:29)

Offline

 

#11 2021-12-05 09:35:07

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

I pushed to Gitlab the import of "container-list" as location.

If you want to add it to tags, you can paste the following code in the DataCrowImport.pm file

Code:

$item->{tags}  = $movie->{'tags-list'};
$item->{tags} .= ", " if ($item->{tags} ne '');
$item->{tags} .= $movie->{'container-list'};

Offline

 

#12 2021-12-05 21:50:23

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

wonderful, modified your suggestion slightly to import to genre, instead of tag:

Code:

$item->{genre}    = $movie->{'genres-list'};
$item->{genre} .= ", " if ($item->{genre} ne '');
$item->{genre} .= $movie->{'container-list'};

will do a quick summary for folks on the data crow forum. really invaluable. thank you again.

is the donation page current? would like to send over a donation but wasn't sure if it's one of the parts of the site that is outdated.

Offline

 

#13 2021-12-05 21:58:52

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

I'm glad that it helped.

If it works, I think the donation will go to Tian who created GCstar. I'm only trying to keep it alive as I didn't find a good alternative.

Offline

 

#14 2021-12-05 22:27:07

nobody
Member
Registered: 2021-11-07
Posts: 17

Re: Help with XML import from Data Crow?

would love to donate to both of you. otherwise many thanks, happy to help test things in the future if you ever need it.

Offline

 

#15 2021-12-29 13:39:48

calibraco
New member
Registered: 2021-12-29
Posts: 2

Re: Help with XML import from Data Crow?

As i was thinking of also stepping over to GCstar from Datacrow.

I have a sugarbag collection, that is now 21,5 GB in backup with DC. It is about 9000 items with 2 scans. Is it possible to transfer this to GCS? And what will be the max amount of items in GCS?

Btw, im no good at programming.

Offline

 

#16 2021-12-31 13:13:13

kerenoc
Administrator
Registered: 2016-03-19
Posts: 477
Website

Re: Help with XML import from Data Crow?

As GCstar doesn't use a database, the max number of items will depend on the available memory on your computer.

I tested the importation of a DataCrow collection of 9000 items : it works but it takes time to process the pictures as they are copied during the import. It also takes time when one chooses the image version of the list :  the thumbnail miniatures need to  be generated.

Currently, GCstar only imports DataCrow collections of books and films. If you have a custom collection type, the model will have to be added to GCstar and the import script will have to be enhanced.

Offline

 

#17 2022-01-05 18:32:23

calibraco
New member
Registered: 2021-12-29
Posts: 2

Re: Help with XML import from Data Crow?

Hmm, ok, i think it will be not worth it to spend time on it, as there are limited collectors of sugarbags.

Thank you for the response.

Offline

 
  • Index
  •  » Support
  •  » Help with XML import from Data Crow?



Should you have a problem using GCstar, you can open a bug report or request some support on GCstar forums.