To address the evaluation protocol used in assessing LiteFallNet's performance for fall detection and its implications for deployment.
Approach:
Evaluation Protocol Analysis: Critiques the method of augmenting fall instances and the subsequent partitioning of datasets, highlighting potential issues with independence in test sets.
Comparison of Splitting Methods: Discusses the impact of record-wise versus subject-wise splitting on model performance and generalization.
Recommendations for Re-analysis: Suggests partitioning subjects before augmentation and evaluating on untouched participants to ensure independence.
Key Findings:
Augmented copies from the same original event can lead to non-independent test sets.
Record-wise splitting may introduce identity confounding.
Subject-wise evaluation is recommended to mitigate confounding effects.
Interpretation:
The performance reported by Owusu et al. should be viewed as specific to the augmentation-first, instance-level protocol.
Limitations:
The datasets used (FallAllD and UMAFall) have a limited number of participants (15 and 17), which may affect the robustness of the findings.
The evaluation protocol may not adequately reflect real-world deployment scenarios.
Conclusion:
A re-analysis with proper subject-wise evaluation is necessary.
