In-Person Poster presentation / top 25% paper

Associative Memory Augmented Asynchronous Spatiotemporal Representation Learning for Event-based Perception

Uday Kamal · Saurabh Dash · Saibal Mukhopadhyay

MH1-2-3-4 #43

Keywords: [ Applications ] [ object recognition ] [ attention ] [ event-based perception ] [ spatiotemporal representation ] [ memory augmented neural network ] [ set processing ] [ event-based camera ] [ Associative Memory ]

[ Abstract ]
[ OpenReview
Mon 1 May 7:30 a.m. PDT — 9:30 a.m. PDT
Oral presentation: Oral 2 Track 6: Applications & Social Aspects of Machine Learning
Mon 1 May 6 a.m. PDT — 7:30 a.m. PDT

Abstract: We propose $\textit{EventFormer}$, a computationally efficient event-based representation learning framework for asynchronously processing event camera data. EventFormer treats sparse input events as a spatially unordered set and models their spatial interactions using self-attention mechanism. An associative memory-augmented recurrent module is used to correlate with the stored representation computed from past events. A memory addressing mechanism is proposed to store and retrieve the latent states only $\textit{where}$ these events occur and update them only $\textit{when}$ they occur. The representation learning shift from input space to the latent memory space resulting in reduced computation cost for processing each event. We show that EventFormer achieves 0.5$\%$ and 9$\%$ better accuracy with 30000$\times$ and 200$\times$ less computation compared to the state-of-the-art dense and event-based method, respectively, on event-based object recognition datasets.

Chat is not available.