#1 2017-07-04 18:51:56

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

So does GCstar work?

Sorry for the provocative subject line, but do have some concerns about GCstar, which is my last attempt at an organized software package before I start to roll my own solutions (at least for "non-Amazon" titles).

My basic ask is to use a barcode scanner to get a list UPCs codes of movies in my collection, then query different websites for information and covers.

I'm just getting started in cataloging both a regular and an adult movie collection. GCstar looked really attractive to me since it was open source, had a good community, and was extensible (from a Plugin point of view) using Perl.

However, I'm having a lot of issues. I initially tried my experiment in OSX and got it running there, but was having Plugin issues -- the Amazon Plugin was fetching some but not all on the information. Figuring it is something quirky I just spent some time experimenting on Ubuntu 14, and I see it is not behaving any differently then on OSX.

Starting with three regular movies picked out of the stack -- Sing (025192366550), Star Wars: The Force Awakens (786936849769), Summer Lover (625828545009). All three of these query fine on Amazon.com using a normal web browser.

Next, the barcode scanner plugin was crashing. This error seems to be benign on Ubuntu, but caused issues on OSX. I implemented a fix which was previously documented here -- http://forums.gcstar.org/viewtopic.php?id=2020, and this fixed my issues on OSX Barcode functionality. But not sure why this didn't end up back in the mainline for release?

So then I can use the Barcode Scanner to get a list of movies. Scanning these barcodes on Amazon (US) resulted in the Director and Cast information being filled in, but not the title, and populated the the catalog as "New Movie". As such cover art, etc was not downloaded. Various fixes here posted at http://forums.gcstar.org/viewtopic.php?pid=28778, but I had to remove the common Amazon Plugin code in 1.7.1 to get these fixes to to show up in the list of useable plugins. Even when I saw them in the list of Plugins to be used by "Fetch Information" button, they still did not complete the Movie entry (i.e. title, cover art) with the entry containing the correct UPC, director, and cast information.

And then after disabling the Amazon Common Plugin code, I no longer had the ability to use Amazon as the default site on the Barcode Import import window. sad

Then there was other weirdness. Using the same partially completed, "New Movie", movie entries, the IMDB plugin returned all the movies from the director (in the "New Movie" item) EXCEPT the one I was trying to catalog.

So this is a nebulous issue, but seems a basic ask. And it seems like Amazon (and, ideally, UPC code import) would be basic supported stuff for a working movie cataloger.

And there are multiple fixes for Amazon out in the wild, that don't seem to work with a barcode scanner (but that I haven't been able to reproduce as working myself).

So is there a future for this software? And in that future, will issues like those listed above be resolved in a normal fix/release cycle, from a directing person or team, via a main build/release system?

My perl skills are a bit rusty, but I'm totally willing to contribute, but would need to see that this software has life and some leaders before that would make sense.

Last edited by crmccluskey (2017-07-04 18:52:49)

Offline

 

#2 2017-07-05 15:35:12

kerenoc
Member
Registered: 2016-03-19
Posts: 93

Re: So does GCstar work?

Hello crmccluskey

Did you checked the latest version of plugins? I'm trying to maintain an updated version of them from the forums and push them to the Test branch of a Gitlab repo, hoping they will be committed some day to the master SVN trunk of GCstar.

Last time I checked the barcode scanner (a long time ago), it seemed to work. But I don't use it as I rather use Shelves, a dedicate app on Android.

It tried a GCstar search for 786936849769 : on Amazon FR the cover and time are correct, on Amazon US, the list of actors, the director and length are ok. It should be possible to make that working again!

Offline

 

#3 2017-07-05 20:00:25

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

Hello kerenoc,

Thank you for the reply.

Well, before I started my testing I performed a "gcstar -u". So I believe they are up to date for what is in 1.7.1.

Would you recommend I sync the lib directory to the content of your Test repo and try it again?

In regards to the Barcode scanner, it doesn't look like the null socket fix is in your Test repo. Per http://forums.gcstar.org/viewtopic.php?id=2020, in GCImportScanner.pm:169 --

Instead of:
$self->{socket}->close;

I used:
if ($self->{socket})
{
  $self->{socket}->close;
}

This stops a null error (for a local, non-network scanner) that prevents the application from taking control on the close of the Import window.

Offline

 

#4 2017-07-06 06:24:24

kerenoc
Member
Registered: 2016-03-19
Posts: 93

Re: So does GCstar work?

I started to track possible modifications of GCstar at the end of last year. I assumed that older bug fixes had been integrated in the Svn repo.

I just pushed the socket fix to Test, you could give it a try.

Offline

 

#5 2017-07-09 03:28:59

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

Hello Kerenoc,

So I just downloaded the test repo and gave it a try, and made some progress.

The null socket fix is in and appears to be working fine.

When using the barcode scanner, I only can choose (5) sites to query information. But in preferences it looks like there is close to 20+ movie sites. Why can't I choose from that list of 20+?

When I do get a positive hit for a found UPC code via the plugin, there is a "Select a Movie" dialog that is not populated with any titles (single or multiple). I hit cancel, but a titles is still imported (likely ok, but a little odd from a UI point of view).

For Amazon (FR), two of the three titles matched (with the Select a Movie dialog noted above for both titles) and successfully imported the title and downloaded the cover art. This might be acceptable, but means that some query responses will be in French (ex: Star Wars: Force Awakens [Blu-ray] [Import anglais]), and a lot a US titles won't be on the Amazon (FR) website.

When I go over to Amazon (US), using the same three movies/UPC, I get (2) titles with a non-populated Select a Movie dialog, and one without, but when I go to see that was imported only the secondary information is present (director, date, etc) for all three titles (which I believe means the UPC code was found, but the harvesting of information was incorrect). The title and cover art are missing.

So unfortunately I still see (3) main issues to overcome.

1) How can I select from other plugins to do an import (I will likely, eventually need to write some of my own to import non-Amazon titles)?

2) Why are the entries in the "Select Movie" dialog missing (I still need to check this on a Linux box to make sure it is not an OS X vs Linux problem)?

3) Why doesn't the Amazon (US) plugin import the title and cover art? DE and UK behave the same as the US plugin -- missing title and cover art, but secondary information is populated -- when the title/UPC is on the DE or UK version of the Amazon website?

Offline

 

#6 2017-07-12 03:55:11

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

Behavior between Linux (Ubuntu 14) and OS X 10.12.5 is the same regarding the Select Movie dialog, so that is good news. The dialog is empty, but behind the scenes something is selected.

I looked briefly at the "Select Movie" dialog, but would need to add more debugging statements to figure out why entries in the dialog aren't being populated.

So looks like the Importer Plugin enumeration for the GCImportScanner is simple enough (line 228). It just looks like most modules haven't listed EAN as an option in the Plugin and/or there is a difference between EAN and UPC. But from a Plugin import methodology I see what is happening.

As for debugging Amazon US, that looks like it might be tricker. I do see http://wiki.gcstar.org/en/websites_plugins for getting started writing plugins.

Is there an applied HOW-TO on how to write an Import Plugin for Movies based on Barcode?

Or, based on experience, is there any of the Movie import modules that are easy to understand (and learn from) as compared to another?

Offline

 

#7 2017-07-12 22:29:50

kerenoc
Member
Registered: 2016-03-19
Posts: 93

Re: So does GCstar work?

Hello,

Hopefully some useful info and good news for you.

1) The plugins declare if the site they connect to is to process EAN search using the getEanField() function. If you know that a site is able to do that, you can write this function, making it return the name of the field to use ("title" for Amazon and DVDfr) and try it (inspiration in GCAmazonCommon.pm).

2) When their associated web site changes, plugins may not be able to collect all information, for example the title of the items. In that case, the Select Movie could contain a list of blank but selectable entries. Corrected for Amazon US.

3) The Amazon US had changed its layout and/or tag enconding. Corrected in latest Gitlab Test version. A bit more complicated than expected as the layout is different when displaying an item from the "Amazon Videos" category or the "TV and Movies" category.

I checked the modifications with GCstar Scanner connecting to an Ubuntu 16.04 server. On Windows/Strawberry Perl, GCstar hangs when listening to the socket. But it works fin if you emulate a local scanner by entering the barcode manually.

Compared to plugins for comics and series, movies plugins are relatively simple. To grab more information it's sometine necessary to connect to multiple web pages (web site with tab such as Allmovie or Allocine). My advice would be to check the HTML tags that are searched by the plugin using for exemple the "inspect" function of Firefox on the web page of an item. Debugging GCstar from Eclipse is also helpful as you can go step by step in the parsing process and check the value of some variables. Some additional info is available in French on a local Ubuntu forum.

There're still some information to parse from Amazon US : region, language, subtitles, If you improve the plugin, it would be great if you could do a "merge request" (GitLab equivalent of "pull requet") on the Plugin_Films_Amazon branch.

Offline

 

#8 2017-07-23 02:17:48

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

Thank you Kerenoc. Very nice.

This looks like a good working base in which to write other scrapers I need. I appreciate the efforts.

If I do work on content that is there I will certainly do a pull/merge request.

Do you know when your Test repo will be merged back into the mainline and released? Just wondering if I should keep a repo myself on the chance that any of the integrated changes are not merged into the mainline.

Thank you again.

Offline

 

#9 2017-07-23 12:07:11

kerenoc
Member
Registered: 2016-03-19
Posts: 93

Re: So does GCstar work?

crmccluskey wrote:

Do you know when your Test repo will be merged back into the mainline and released?

No, I think the original author as little time for the GCstar maintenance.

Furthermore, the SVN repo was hosted on a site that closed in May. My Gitlab repo seem to have now the latest updates on the scripts.

A consequence is that GCstar may no be available as a package of some Linux distributions, possibly reducing the user base.

Offline

 

#10 2017-08-12 02:14:46

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

With the current support (thank you Kerenoc), I have CGstar working to the point that I can start importing more items in the collection.

To that end I have written my own plugin. It currently is importing most data types for film that I need (title, images, date, synopsis, actors, genre, and director). However, I can't seem to get it to populate the item in the Details Tab -> Media or Details Tab -> Video Format fields. It is the assumption that the Media field is populated with the "format" field definition.

1. I have confirmed that I'm extracting the correct values from the web page.

2. In sub new, I'm initializing the field --
        $self->{hasField} = {
            title => 1,
            date => 1,
            director => 1,
            actors => 1,
            synopsis => 1,
            format => 1,
            genre => 1
        };

3. And I'm assigning it (explicit for testing) with --
       $self->{curInfo}->{format} = 'BLU-RAY';

But I still can't seem to get the UI to show it. Any ideas on how to debug this further?

Continuing thanks,
Chris

Offline

 

#11 2017-08-13 07:47:41

kerenoc
Member
Registered: 2016-03-19
Posts: 93

Re: So does GCstar work?

Hello

Thank you for contributed to the evolution of GCstar.

I investigate and found that the film model file, the "format" and "video" field are not supposed to be imported. If some plugins start to get that information from web site, that may be changed. One has to be careful though as some users may fetch generic information from the web and enter specific information manually later.

I also started to improve EAN support for some collections (not yet on Gitlab) : the scanner is working with EPC/EAN but this information was not retained.

Attached is a propose modification of GCfilms.gcm

Offline

 

#12 2017-08-13 16:46:03

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

I totally missed the "imported" field. cool

If I wished to respect a previously entered format I could just do something like the following --

if !($self->{curInfo}->{format})
{
     $self->{curInfo}->{format} = <format>;
}

Correct?

Offline

 

#13 2017-08-13 17:21:58

crmccluskey
Member
Registered: 2017-05-21
Posts: 13

Re: So does GCstar work?

Well, the import works now, but it doesn't respect any value that may exist prior. There will be times that I would want to set it to DVD-R manually (even though the site says it's DVD), and only import the format if one doesn't exist. Is there an accessor for reading existing fields from the plugin?

Last edited by crmccluskey (2017-08-13 17:29:04)

Offline

 

#14 2017-08-14 07:46:09

kerenoc
Member
Registered: 2016-03-19
Posts: 93

Re: So does GCstar work?

To preserve some fields when importing, the best way may be to change in the preferences setting dialog the configuration of the web sites to use. By selection "Many sites (per field)", one can choose a web site for each type of field or exclude a field from the fetch.

Offline

 



Should you have a problem using GCstar, you can open a bug report or request some support on GCstar forums.