input_manager: don't ignore double click event when clicking inside device

Signed-off-by: Yu-Chen Lin <npes87184@gmail.com>