You can use css-only, too. Make your link image and the hover image below it and note the width and 1/2 height of this combined image. Links are often displayed as list items, so try them in an unordered list.
As for the css, if each link img. is a diff. size, you'll have to create a unique id for each. For the a:link and a:visited states, the background image position will be different then the a:focus and a:hover ones - which will be moved up by the amt. of px in 1/2 the image's height.
[Sorry not to write it all out, but I was just checking-in before bedtime!
If you still need some help tomorrow, maybe there'll be some time ]