Update 2014-04-25: This post has been superceded by my new method.
I have a personal server (tolaris.com), a large collection of eBooks, and several Android devices. I like to read books on the go, without the hassle of actually carrying dead trees. But I don’t want to manage local copies of ebooks on my phone or tablet. What I really want is a personal library, accessible anywhere, from which I can download my books at any time. And now I have that.
What you’ll need:
- Calibre, to manage your eBook library. Calibre is awesome. It supports all the formats you are likely to care about and can convert between them, and stores and manages your books in a way you can easily access with a file browser.
- Calibre2opds, which generates OPDS and HTML catalogs from the metadata of Calibre. OPDS is an interchangeable format for describing and sharing books – such as in a personal library.
- A personal server with an HTTP server, such as Apache. You can use a real server or a virtual machine as long as it has a public IP address you can access. Or you could use your desktop PC, accessible only within your private LAN when you’re at home. If you don’t want to run your own server, you could even use Dropbox.
- eBook software with OPDS support, such as FBReader. I read primarily on my Android devices, a Nook Color and an HTC Desire HD, both running community-built ROMs. You can get FBReader on the Market.
How to do it:
Install Calibre
Calibre is easy to install (apt-get install calibre on Ubuntu), and easy to use. I won’t cover it here. Install it, add some eBooks to your library, and close the application.
Install and configure Calibre2opds
Calibre2opds supports both a GUI mode, which I use only to configure it, and a command-line mode, which is handy for updating your library from a script. Unfortunately, the CLI mode has been broken since 2.4-beta4, and isn’t expected to be fixed until the next major release. So I continue to use that release. The developers no longer provide 2.4-beta4, so download it here.
Unpack the zip file somewhere, and open a terminal. Run it for the first time:
sh ./rungui.sh
Configure the following settings on the “Main options” tab:
- Set “Database folder” to the root of your Calibre library directory.
- Leave “Destination folder” blank.
- Uncheck “Copy Catalog to database folder”.
Now press “Save”, then “Generate catalogs”. This can take several minutes to complete if you have a large library.
If you have run Calibre2opds once and saved your settings, you can run it without the GUI using:
sh ./run.sh
Upload to the server
Now upload your library and catalog to your server. If you run your own server you probably know how to do this. If not, see the Dropbox method above.
I use a simple script with rsync to upload ~/Library to a private area of my web server. Here is a very simple script that you can run after you add new books to Calibre.
#!/bin/sh # if Calibre metadata is newer than the OPDS index, update the catalog if [ "$HOME/Library/metadata.db" -nt "$HOME/Library/index.xml" ] ; then cd ~/calibre2opds/ sh ./run.sh else echo "Calibre has not been run since last update, skipping calibre2opds." fi rsync -av --delete $HOME/Library/ webuser@example.com:~/public_html/my_personal_library/
Once this is done, you will have a HTML catalog you can browse:
Install and configure FBReader
- Install FBReader from Android Market and open it.
- Press the Menu key, then select “Network library
- Press the Menu key, then select “Add catalog”
- Select “Enter URL manually”
- Enter the URL to your library, appended with “_catalog/index.xml”. For example, “http://www.example.com/my_personal_library/_catalog/index.xml”.
- Press OK twice. Your catalog will now appear in the list.
You can now browse your catalog within FBReader, select any book, and download it, and read it without navigating out again.
If you are an iPhone user, consider MegaReader, available free in the App Store. Configuration is similar.
Updating the library
Once you have all this set up, updating your library is easy. Just add the book in Calibre, run calibre2opds, and sync the files to your server. It’ll be available in FBReader as soon as your access the network library.
-
Can you see an advantage/disadvantage in using “calibre2opds”, instead of installing calibre and using it’s built in opds compatible content server along with apache?
http://manual.calibre-ebook.com/server.html
I’m considering using “owncloud” along with the calibre content server.
-
Owncloud is cool. :)
The difference, aside from the appearance of the pages, is that using Calibre’s content server requires supporting a running instance of Calibre on your web server. Using Calibre2opds, you get html and xml. You don’t need python or any of Calibre’s dependencies. You don’t have to ensure that Calibre is or remains running. I have nothing against python, but I don’t think installing a GUI on a server is a good idea.
-
Great tutorial! But you actually keep your library in two places?
I used to use Calibre + Calibre OPDS, but the setup is bit complicated and regenerating the catalog every time manually is annoying. So I end up with a simple Web service I programmed, catalog.im. It helps me to organize all my ebook collections and it gives a up-to-date OPDS feed. All I need to do is to upload my ebooks, and these ebooks will appear in the library and the OPDS feed.
I would like to invite you to try it out :D
OK, I admit, it’s a promotion :ashamed:
-
I don’t mind the promotion, but this is a cloud service. The point of my solution is to store my books myself, on my infrastructure. I look forward to you releasing the source behind catalog.im, so we can do the same.
I use an rsync script to push my library from my local device to my web server. I’d be quite happy if Calibre had a web-based front end that didn’t require me running X on the server.
-
Hi,
Using Calibre Windows Desktop app, I have created my personal e-books library on Google Drive inside a public folder called Books.
I have created 2 separate libraries. Fiction and Non Fiction. Hence within the Books folder of my GDrive account, I have 2 sub folders. Fiction and Non Fiction.
I then used Calibre2OPDS v3.2 to create my OPDS catalog. One each for the above two sub folders within the sub folders that they are for. Fiction and Non Fiction.
I then tried to add GDrive created links of index.xml and index.html to Mantano 2.4.11 OPDS libraries. I could add both links but Mantano came up with a blank screen every time I tried to access my catalog using either of the links. I did this for both Fiction and Non Fiction catalogs separately.
(I experienced all of the above issues using Dropbox too)
I then installed Calibre Cloud Pro app on my Android Note 2 4.3JB. I pointed this app to the Book folder of my GDrive account where the library Created by Calibre Desktop app currently resides. It only showed some books from the sub folder Non Fiction and none from the Fiction folder thus showing only the partial library.
Am I doing something wrong? Please help me so that I can either continue using the Calibre Cloud Pro or get Mantano to read from my OPCDS catalog residing on GDrive and created as per above. In theory it’s just what I need. In practice it is not working out that way.
Another thing, when using Calibre Cloud Pro app, the following performance failures were observed by me every time:
1. Direct download of an e-book failed to my Note 2 continuously.
2. ‘Open with’ took me back to the book description screen. It did not give me an option of an app to open my e-book with.
3. ‘Add to Mantano’ functioned exactly like 2 above. No books were added to Mantano Premium that I am using for now.
I look forward to your response.
Kind regards,
Rishi Saran.
-
Sorry, Rishi, I can’t help you. I’m no longer using Calibre2OPDS, and I’ve never used GDrive or Dropbox for sync. Nor have I used Calibre Cloud Pro. In fact, I now use Calibre+ownCloud+COPS:
http://www.tolaris.com/2014/04/25/a-better-way-to-share-ebooks-or-my-opds-library-version-2-0/
My first suggestion is to test against another eBook reader with OPDS support like FBReader. Every reader has odd quirks of what it can support and what it cannot. If you find a solution, please let me know.
-
I’m amazed, I have to admit. Seldom do I come across a blog that’s
both educative and interesting, and without a doubt, you have hit
the nail on the head. The problem is something
that too few men and women are speaking intelligently
about. I am very happy that I came across this in my search
for something concerning this. -
If you really like Calibre but doesn’t trust any cloud service, you can host your own Calibre catalog in your own webserver (on your Nas or elsewhere). That’s why I coded COPS (Calibre OPDS PHP Server)
You can get more information and see a demo here : https://blog.slucas.fr/en/oss/calibre-opds-php-server
Of course if you don’t have a webserver, dropbox is the way to go.
-
That is slick, Sébastien. Especially the search feature. I’ll give it a test on my own library.
This effectively skips steps 2 and 4 above, and means I only have to run Calibre, then sync its Calibre’s files.
-
I totally don’t get that. Where is there any saving of effort running COPS rather than calibre2opds? I’m half-way—I have my Calibre library synced to my server with owncloud, but then I can’t decide between COPS and calibre2opds, and you’re not helping :-)
-
calibre2opds method:
1. Add book to Calibre and close it.
2. Run Calibre2OPDS to generate OPDS catalog.
3. Sync files to server.COPS method:
1. Add book to Calibre and close it.
2. Sync files to server. -
See the top of the post. This post has been superceded by my new method:
http://www.tolaris.com/2014/04/25/a-better-way-to-share-ebooks-or-my-opds-library-version-2-0/
-
I DID see the top of the post, and I’d read the other version. I was asking why you felt it was so much simpler (you said it eliminated two steps, and I couldn’t see that—especially since you said “steps 2 and 4” but didn’t number your steps).
So, really, it eliminates one step, but you’re right, using COPS _is_ simpler—but not as much simpler as you suggested. I initially thought calibre2opds would also save me having to install Apache and PHP on the server, but then I realized those are both necessary for owncloud.
otoh, I have 3000+ books in calibre (and it still hasn’t finished syncing…), and only 1200 with e-books attached. calibre2opds can create pages for only those 1200. I don’t see that COPS does that (which is exactly why I wanted to get away from using the Calibre content server—I’ve got no interest in having books on the OPDS server that can’t be downloaded).
-
I see your point; that wasn’t clear. Either I made a mistake, or I’ve edited the post since and can’t recall the missing 4th step. History doesn’t enlighten me, so I think it’s the former. Sorry about that!
I have since stopped using owncloud, so I now use Calibre + an rsync script + COPS. If you wanted to do the same, you could just exclude non-epub files from the rsync and save yourself the effort of syncing them.
-
Yeah, I’m leaning that way. I’m just not finding owncloud reliable enough. It spends a lot of time doing things I don’t care about (like checking that nothing changed on the server side—for this, at least, nothing ever will!), and that interferes with knowing if everything has been copied from the client yet. At least with rsync I’ll always know when the latest sync occurred.
There’s the gem of what I want in the exclusion idea, but it’s not quite what I want. I don’t have a problem with getting the PDFs and MOBIs (though calibre2opds allowed me to have just the one with highest priority; it’s having books in the catalog with none of those that is the problem.
-
If I understand correctly, you want to not display the catalog entries lacking EPUB files in COPS (or something like it). If so, two ideas:
1. Modify COPS to check before displaying the catalog entry. It’s PHP, and should be easy to add. If you do so, give it a config option that defaults to off, and Sebastian (the author) will probably accept the patch.
2. Use 2 Calibre databases, one with EPUB and one without.Or, consider reading (on Android) with FBReader, which can handle those other formats.
-
The “other formats” are not a problem. I want to suppress the ones that have _no_ formats (calibre is my book database–it has every paper book I own, a lot I’ve just read, AND all my e-books.) Generally, if I have a PDF or any other e-format, I’ll have converted it to ePub, anyway, so really only ePub matters, but I don’t have a problem with having all of the other e-formats. I just want to suppress the paper-only versions. I still don’t have COPS installed, so I might be completely wrong, and maybe it already does that (the Calibre content server certainly doesn’t), but that’s not what I understand from the documentation I’ve read (the demo doesn’t help–it only shows e-books, but maybe that’s all that are in his sample).
Anyway, I figured it would be simple enough to modify COPS, so I’ll go that route.
-
Installing COPS is trivially easy. Unpack tar, edit config file, test. Give it a go. Try testing if non-present books are even displayed.
COPS has this option:
$config['cops_prefered_format'] = array ("EPUB", "PDF", "AZW3", "MOBI", "CBR", "CBZ", "CHM");If the above doesn’t help, you could add a $config[‘cops_show_prefered_formats_only’] option with just a few lines of PHP.
-
Hmmm. Not _trivially_ easy. It’s a long time since I hand-installed a PHP server (though I installed two PHP servers from apt just this afternoon), and there’s obviously something in the apache config I’ve forgotten.
-
So, not quite trivial, but not so bad as all that. Except for forgetting to reload Apache, the only thing wrong was that I didn’t have php5-intl installed, and it’s not mentioned in his FAQ. It does include books with no formats, so I’ll see what I can do about that.
-
-
-
-
-
-
-
-
-
-
-
-
-
20 comments
Comments feed for this article
Trackback link: https://www.tolaris.com/2012/02/13/my-opds-library/trackback/