Fedora, tracker RSS miner and sqlite FTS support

Hi readers, as you might know Allan Day gave us initial mockups for new GNOME project – News. This post is not about first preview working application, this post is about how we started writing application and how we helping other GNOME and Fedora projects :)

GNOME News mockup

With Vadim Rutkovsky we started working on this app yesterday’s night. We did choose to use tracker as storage backend and libgrss as our fetcher engine.

But we found some issues with them:

  • libgrss ships its .gir and .typelib in wrong directory – FIX. I spotted this when packaged libgrss, not when we started coding
  • libgrss introspection a bit wrong because it’s provides us gpointer instead of GrssFeedItem – FIX
  • tracker wants older name of libgrss (libgrss-0.6), but API/ABI have not broken and 0.5 version doesn’t work for us – FIX

Okay, we fixed this in upstream, cherry-picked changes to Fedora, but after enabling FTS support in Fedora’s sqlite – tracker doesn’t work on my laptop. I’ve spent a lot of time with trying to decide what happens there.

I gave SSH to my laptop for Carlos Garnacho and he realized that FTS support at compile-time was not enabled and tracker tries to load its own FTS, but actually FTS module from sqlite already loaded, so we got trying to load 2 the same modules (from sqlite and from tracker) twice.

Then I started rebuilding tracker in Fedora and got the same FTS not enabled at compile-time. The problem was with broken CFLAGS/LDFLAGS in m4 macro. In parallel I also found that /usr/bin/tracker-control has no executable flag and there are no tracker-compat binary. All those fixed in 1.5.0. Finally I updated tracker in Fedora to 1.5.0 with enabled FTS and RSS support – Koji build.

Feel free to join our channel in IRC: #gnome-news at GimpNet


Carlos debugging tracker on my laptop:

(gdb) b sqlite3MisuseError
Function "sqlite3MisuseError" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (sqlite3MisuseError) pending.
(gdb) r
Starting program: /usr/libexec/tracker-store 
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.21.90-17.fc23.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7fffec0e7700 (LWP 26995)]
[New Thread 0x7fffeb62b700 (LWP 26996)]
[New Thread 0x7fffeae2a700 (LWP 26997)]
[New Thread 0x7fffea629700 (LWP 26998)]
[New Thread 0x7fffe9e28700 (LWP 26999)]
[New Thread 0x7fffe9627700 (LWP 27000)]
[New Thread 0x7fffe8e26700 (LWP 27001)]

Breakpoint 1, sqlite3MisuseError (lineno=lineno@entry=114597) at sqlite3.c:131133
131133	SQLITE_PRIVATE int sqlite3MisuseError(int lineno){
Missing separate debuginfos, use: dnf debuginfo-install bzip2-libs-1.0.6-16.fc23.x86_64 dbus-glib-0.104-3.fc23.x86_64 dbus-libs-1.9.16-2.fc23.x86_64 dconf-0.24.0-2.fc23.x86_64 elfutils-libelf-0.163-1.fc23.x86_64 elfutils-libs-0.163-1.fc23.x86_64 GConf2-3.2.6-15.fc23.x86_64 glib2-2.45.3-2.fc23.x86_64 libattr-2.4.47-12.fc23.x86_64 libcap-2.24-8.fc23.x86_64 libffi-3.1-8.fc23.x86_64 libgcc-5.1.1-4.fc23.x86_64 libgcrypt-1.6.3-5.fc23.x86_64 libgpg-error-1.19-2.fc23.x86_64 libicu-54.1-1.fc22.x86_64 libselinux-2.3-11.fc23.x86_64 libstdc++-5.1.1-4.fc23.x86_64 libuuid-2.26.2-1.fc23.x86_64 pcre-8.37-2.fc23.x86_64 systemd-libs-222-1.fc23.x86_64 xz-libs-5.2.1-3.fc23.x86_64 zlib-1.2.8-8.fc23.x86_64
(gdb) bt
#0  0x00007ffff45386d0 in sqlite3MisuseError (lineno=lineno@entry=114597) at sqlite3.c:131133
#1  0x00007ffff4538922 in createModule (db=db@entry=0x5555557aad18, zName=zName@entry=0x7ffff45a95bb "fts4aux", pModule=pModule@entry=0x7ffff47c9d40 <fts3aux_module>, pAux=pAux@entry=0x0, xDestroy=xDestroy@entry=0x0) at sqlite3.c:114597
#2  0x00007ffff4538a08 in sqlite3_create_module (db=db@entry=0x5555557aad18, zName=zName@entry=0x7ffff45a95bb "fts4aux", pModule=pModule@entry=0x7ffff47c9d40 <fts3aux_module>, pAux=pAux@entry=0x0) at sqlite3.c:114637
#3  0x00007ffff45a77ab in openDatabase (db=0x5555557aad18) at sqlite3.c:139434
#4  0x00007ffff45a77ab in openDatabase (db=0x5555557aad18) at sqlite3.c:5800
#5  0x00007ffff45a77ab in openDatabase (zFilename=<optimized out>, ppDb=ppDb@entry=0x5555557a4b00, flags=6, zVfs=zVfs@entry=0x0) at sqlite3.c:65318
#6  0x00007ffff45a7c05 in sqlite3_open_v2 (filename=<optimized out>, ppDb=ppDb@entry=0x5555557a4b00, flags=<optimized out>, zVfs=zVfs@entry=0x0) at sqlite3.c:130937
#7  0x00007ffff7b8e1e5 in tracker_db_interface_initable_init (error=0x7fffffffda50, db_interface=<optimized out>) at tracker-db-interface-sqlite.c:940
#8  0x00007ffff7b8e1e5 in tracker_db_interface_initable_init (initable=<optimized out>, cancellable=<optimized out>, error=0x7fffffffdba0)
    at tracker-db-interface-sqlite.c:1022
#9  0x00007ffff4fc06fa in g_initable_new_valist () at /lib64/libgio-2.0.so.0
#10 0x00007ffff4fc07b6 in g_initable_new () at /lib64/libgio-2.0.so.0
#11 0x00007ffff7b8f105 in tracker_db_interface_sqlite_new (filename=filename@entry=0x5555557a9e70 "/home/garnacho/.cache/tracker/meta.db", error=error@entry=0x7fffffffdbe0)
    at tracker-db-interface-sqlite.c:1746
#12 0x00007ffff7b90429 in db_interface_create (type=TRACKER_DB_METADATA, error=0x7fffffffdbd8, create=<synthetic pointer>) at tracker-db-manager.c:324
#13 0x00007ffff7b90429 in db_interface_create (error=0x7fffffffdbd0) at tracker-db-manager.c:352
#14 0x00007ffff7b90429 in db_interface_create (db=<optimized out>, error=0x7fffffffdc30) at tracker-db-manager.c:374
#15 0x00007ffff7b9158a in perform_recreate (error=0x7fffffffdc28) at tracker-db-manager.c:753
#16 0x00007ffff7b9158a in perform_recreate (first_time=<optimized out>, error=0x7fffffffdca0) at tracker-db-manager.c:838
#17 0x00007ffff7b91df1 in tracker_db_manager_init (flags=flags@entry=TRACKER_DB_MANAGER_REMOVE_CACHE, first_time=first_time@entry=0x7fffffffde94, restoring_backup=restoring_backup@entry=0, shared_cache=shared_cache@entry=0, select_cache_size=select_cache_size@entry=100, update_cache_size=update_cache_size@entry=100, busy_callback=0x555555566220 <_tracker_status_callback_tracker_busy_callback>, busy_user_data=0x5555557a1920, busy_operation=0x55555556efe0 "Initializing", error=0x7fffffffdea0) at tracker-db-manager.c:1161
#18 0x00007ffff7b82299 in tracker_data_manager_init (flags=TRACKER_DB_MANAGER_REMOVE_CACHE, test_schemas=0x0, first_time=0x7fffffffdfac, journal_check=1, restoring_backup=0, select_cache_size=100, update_cache_size=100, busy_callback=0x555555566220 <_tracker_status_callback_tracker_busy_callback>, busy_user_data=0x5555557a1920, busy_operation=0x55555556efe0 "Initializing", error=0x7fffffffdfd0) at tracker-data-manager.c:3741
#19 0x000055555555eb4b in main (args_length1=1, args=0x7fffffffe0f8) at /home/carlos/Source/gnome/tracker/src/tracker-store/tracker-main.vala:263
#20 0x000055555555eb4b in main (argc=<optimized out>, argv=<optimized out>) at /home/carlos/Source/gnome/tracker/src/tracker-store/tracker-main.vala:150
(gdb) up
#1  0x00007ffff4538922 in createModule (db=db@entry=0x5555557aad18, zName=zName@entry=0x7ffff45a95bb "fts4aux", pModule=pModule@entry=0x7ffff47c9d40 <fts3aux_module>, 
    pAux=pAux@entry=0x0, xDestroy=xDestroy@entry=0x0) at sqlite3.c:114597
114597	    rc = SQLITE_MISUSE_BKPT;
(gdb) l
114592	  int nName;
114594	  sqlite3_mutex_enter(db->mutex);
114595	  nName = sqlite3Strlen30(zName);
114596	  if( sqlite3HashFind(&db->aModule, zName) ){
114597	    rc = SQLITE_MISUSE_BKPT;
114598	  }else{
114599	    Module *pMod;
114600	    pMod = (Module *)sqlite3DbMallocRaw(db, sizeof(Module) + nName + 1);
114601	    if( pMod ){
(gdb) p zName
$1 = 0x7ffff45a95bb "fts4aux"
(gdb) q
A debugging session is active.

	Inferior 1 [process 26989] will be killed.

Quit anyway? (y or n) y

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments are closed.

  1. Ryan T. says:

    All of those bugs in Fedora for so long… No wonder so many people are moving to containers. Fedora QA is absolute dogshit.