The research on image based scene representation for visualisation done at the Czech Technical University shall be overviewd. First, the experiences with image synthesis using a simple interpolation will be discussed. Then, the problem of selection of a sparse set of representative reference views will be addressed and one possible approach will be shown. Finally, recent work on image extrapolation dealing with occlusions will be presented.