Working on YOURLS 1.6 : the next DB structure

What’s up, gents? I’ve begun working on the next iteration and here’s what I’m up to.

YOURLS 1.5 : DB suckage

As everybody knows, the current database design in YOURLS is dumb and very inefficient. Its biggest flaw is that the keyword (ie short url) is repeated in both the URL and the LOG tables, making it absurdly difficult to update a short url without losing all historical data.

Another design decision I regret was to store stuff in case they would be useful to anyone, such as user agents in the log table. Since there’s no core feature using that info and I probably won’t implement any, this should not be there. There’s a plugin API for this.

There are a couple of awesome features I want to work on for YOURLS 1.6, one of them being the ability to store arbitrary data associated to any short URL — think url meta data, the way WordPress does it.

YOURLS 1.6 : previous thoughts

I’ve been pondering about the next DB schema for a very long time now, trying to think about the most state-of-the-art structure I could come up with (given my overall blatant lack of skill for DB things) and addressing all possible future features and current issues. I’ve proposed stuff and a few people have been kind enough to comment. But all this had one weakness : it was complicated and scaring me. Seriously :)

YOURLS 1.6 : smarter yet simple (at least I hope so)

I decided to make up my mind once for good and here is how I see things in YOURLS 1.6:

The main URL table will be properly normalized with a URL id to stop repeating the keyword across tables. I could probably go further in normalization (storing the long URL and title somewhere else, as I first thought) but that’s where I think it becomes too much trouble and hassle for too few benefits.

The URL_META table will store anything you’d like about a particular short URL and will be used by plugins: some tags, a note, a mime type to handle redirection differently, anything.

The LOG table will be trimmed down a bit, which — disk wise — should be beneficial to sites with lots of hits, and properly. Again here I could probably go further and normalize the referrer information, but what bothers me then is the number of DB queries needed for each short URL redirection, which I want to keep at a very minimum.

The LOG_META table, just as its meta sibling, will store anything you’ll want to store about a hit: the user agent, some cookie info, anything.

No big change in the OPTIONS table, just an autoload parameter so plugins will be able to store anything without loading that every time in RAM.

So, that’s it. If you have any thought or any “zomg dude don’t, terrible decision” warning to share, please do. We’ll see later for other DB novelties such as log archives or further optimization. I’d rather stop pondering and start coding :)

Short URL to this post: http://yourls.org/6g

YOURLS 1.5.1 Released

I’m happy to announce that YOURLS 1.5.1 is available and waiting for you. Get it quick, the first 1,000 installs get 20% off the regular price!

What’s new

YOURLS 1.5.1 is a “minor” release, as in “there are no really new important features but lots of improvements and bug fixes”. Some very cool stuff nonetheless, which are:

  • You can now encrypt your passwords in the config file instead of storing them in clear text. See Using encrypted salted passwords
  • YOURLS stat pages now uses the new Google Visualization API. See that page for example.
  • Stat pages are much more intelligently generated, SQL wise. That alone makes it worth upgrading
  • The API is now JSONP compliant, for all you Javascript nerds
  • Drop a favicon in the /user directory and YOURLS’ll use it

You can check a slightly more detailed changelog.txt and, if you’re really wanting to keep an eye on development, follow @YOURLS (for more general news about YOURLS, and other stuff, you’ll want to follow Ozh instead — much more entertaining :)

15 second updating

That’ll be a quick update: just upload the new archive to overwrite all your files, and that’s it. No scary update page to remind you that everything should go fine but you should backup everything just in case. Note that I’m not saying that you shouldn’t backup regularly anyway :)

Oh, and while you’re downloading the latest archive, spend a second or two on yourls.org and give that “social” icons the spanking they deserve! The Showcase section has been recently updated and, man, there are quite a few cool things around made with or for YOURLS!

What’s next

I shall now start to make up my mind about the next database schema for 1.6 and work on the next iteration. This will imply lots of DB changes and I’m truly scared because this is my sucky area :)

I’m also currently pondering over moving the project on Github or not, see previous blog post on that subject.

OK, those last two paragraphs make me sound like I just can’t make any quick decision, but, Ma, I swear this isn’t what it looks like! :)

Now be a good boy and tell a friend to update while you’re updating, will you?

Short URL to this post: http://yourls.org/5t

Should YOURLS be on GitHub instead?

I’m regularly asked if there’s a chance that YOURLS be hosted on GitHub instead of Google Code as it is now.

Sure, GitHub is sexier, the web interface is much slicker, trendy and all. On the other hand, Google Code is here to stay forever or so, I know SVN, the YOURLS project has all its ticket and source history there and, despite being a bit raw sometimes, the interface and the service get the job done.

One argument I’ve been given a couple of times is “you’d get more contribution on GitHub”, but I’m not convinced: why would it be easier on GitHub (just because you just click “Pull Request” and save the minor step of generating a diff file?). The counter argument that easily comes to my mind is: spending hours on learning new tools and migrating the project and its history (issues and commits).

But maybe I’m not seeing the big picture and there are factual reasons why hosting on GitHub would be more beneficial to the YOURLS project. If you believe that’s the case, please do comment and convince me :)

Short URL to this post: http://yourls.org/5k

Forums are gone

I decided to remove the forums on this site. Long story made short: I’m uberfedup with dealing with spam and I’d rather do something productive with my too rare free time (and, to be honest, I’m surprised it lasted so long before I decided this)

So long, chinese spammers. Sorry for the (very few) users who felt this was useful.

Short URL to this post: http://yourls.org/4n

How to use Twitterfeed with your custom YOURLS URL shortener

For many, Twitterfeed means you have to use bitly. See, it is very easy to set up Twitterfeed to use your own YOURLS install and effortlessly publish tweets from a custom RSS feed using your own URL shortener. For instance, I do this on the @yourls account, where every source commit on the project gets tweeted.

Twitterfeed + custom URL shortener

To do so, create a new feed on Twitterfeed, and in the Advanced Settings, check “Post Link” and pick “Custom” as “Shorten link through” option. You’ll be given an input field to define your “Custom endpoint”. In this field, enter the URL of your YOURLS API, like so:

http://sho.rt/yourls-api.php?signature=123456&action=shorturl&format=simple&url=%@

The important bits in this URL endpoint are:

  • the URL of your YOURLS API, obviously
  • your own secret signature
  • url=%@, because Twitterfeed will replace “%@” with the URL to shorten

Twitterfeed Advanced Settings

Have fun!

Short URL to this post: http://yourls.org/3o

YOURLS goodies: forums, nightly builds and hook list.

There are a few things I’ve discreetly mentioned over the past weeks, either on Twitter (follow me!) or through the project wiki pages, that are well worth an official blog post: forums, nightly builds and hook list.

1. Forums!

Some time ago, I’ve setup a few YOURLS forums. There are currently 3 forums:

  1. General discussion: show off your cool public interface or personalized YOURLS index page with stats and stuff, discuss anything, bounce ideas.
  2. Plugins: anything related to plugins. Introduce yours and ask for feedback, suggest plugin ideas.
  3. Help Wanted!: if you have a problem and need someone to lend a hand, try here. The project issue tracker is not suitable for personal support requests (its goals are to report defects or suggest feature requests, not troubleshoot your own unique personal .htaccess issue), but this forum is. Maybe some freelancers will even get a couple paid gigs here too?

If these forums do meet a need and become active, I may ask a few volunteers to help moderate it in the future.

2. Nightly builds!

There are now YOURLS nightly builds generated, well, every night. This is aimed at users who want to update their install (either to code plugins or to fix bugs without waiting for an official package) and don’t know how to update YOURLS with SVN.

For the record, unless stated otherwise the current trunk version of YOURLS is safe to run on a live site. For instance, I run it on yourls.org and also my personal URL shortener, ozh.in, and things have always been fine.

Remember: when coding plugins, it’s recommended to use the latest available, that is either running trunk or installing a nightly build. Which leads to me third point:

3. Hook list!

YOURLS plugin coders will be pleased to know that there is now an exhaustive and always up to date list of YOURLS actions and filters. This list is generated every night against the current trunk version, thanks to the help of a smart user who crafted a neat perl script.

Short URL to this post: http://yourls.org/39

Hot: APC Cache plugin for YOURLS

I’m particularly excited by this one: Ian Barber got his hand dirty and wrote the first release of an APC Cache plugin for YOURLS.

In case that’s Greek to you, APC stands for Alternative PHP Cache and is a PHP extension that provides an opcode cache for PHP: it caches data and compiled code in memory. From the readme:

This plugin is designed to remove a lot of the database traffic from YOURLS, primarily the write load from doing the logging and click tracking. We have attempted to strike a balance between keeping most information, but spilling it in some cases in the name of higher performance.

If you have a server capable of running APC, especially if you’re running a popular URL shortener, please give the plugin a try and send Ian some feedback!

Short URL to this post: http://yourls.org/1q

Preventing abuse of your public YOURLS install

A lot of you, beloved users, are using YOURLS to set up public shorteners. Sooner or later you all face the same problem: abuse by spammers, sometimes until your hosting provider shuts down your site. Hopefully, thanks to the plugin API, being abused by spammers is not inevitable destiny and plugins can address that issue.

I know at least 2 plugins on this topic:

If you made a plugin on this topic, be sure to let me know (remember: DON’T MODIFY core, make a plugin instead!!)

Short URL to this post: http://yourls.org/11