The image which is displayed as the 'cover' for each album is indicated by a yellow icon in the top left of a selected image in an album.
To change which image is used as the cover, select a new image in the album and in the info sidebar on the right you'll see an option to 'Set as cover image'.
Click this button and the new image will be set as the cover image and you'll see this confirmed via a pop-up and indicated by the yellow star moving to the new image.