Over the last few years, a very active field of research has aimed at exploring new data-driven and learning-based methodologies to propose computationally efficient strategies able to benefit from the large amount of observational remote sensing and numerical simulations for the reconstruction, interpolation and prediction of high-resolution derived products of geophysical fields. In this paper, we investigate how they might help to solve for the oversmoothing of the state-of-the-art optimal interpolation (OI) techniques in the reconstruction of sea surface height (SSH) spatio-temporal fields. We focus on two small 10°×10° GULFSTREAM and 8°×10° OSMOSIS regions, part of the North Atlantic basin: the GULFSTREAM area is mainly driven by energetic mesoscale dynamics, while OSMOSIS is less energetic but with more noticeable small spatial patterns. Based on observation system simulation experiments (OSSE), we used a NATL60 high resolution deterministic ocean simulation of the North Atlantic to generate two types of pseudo-altimetric observational dataset: along-track nadir data for the current capabilities of the observation system and wide-swath SWOT data in the context of the upcoming SWOT (Surface Water Ocean Topography) mission. We briefly introduce the analog data assimilation (AnDA), an up-to-date version of the DINEOF algorithm, and a new neural networks-based end-to-end learning framework for the representation of spatio-temporal irregularly-sampled data. The main objective of this paper consists of providing a thorough intercomparison exercise with appropriate benchmarking metrics to assess whether these approaches help to improve the SSH altimetric interpolation problem and to identify which one performs best in this context. We demonstrate how the newly introduced NN method is a significant improvement with a plug-and-play implementation and its ability to catch up the small scales ranging up to 40 km, inaccessible by the conventional methods so far. A clear gain is also demonstrated when assimilating jointly wide-swath SWOT and (aggregated) along-track nadir observations.