Quadro-W learning for human behavior prediction in an evolving environment: a case study of the intelligent butler technology

In recent years, with advances in hardware devices (e.g., sensors and microprocessors), increases in the maturity of software technology, increases in popularity of the Internet, and decreases in the costs of technologies, embedded systems have been widely used in various applications, including in crop growth monitoring, commodity defect detection, transportation system management, and vital signs monitoring. However, to obtain sufficient information for analysis, various sensors must usually be installed in environments. This requirement can cause many problems, such as changes in the original environment because of the installation of hardware devices, a time-consuming process for setting up the system, high costs given the use of many hardware devices, and difficulty in system maintenance. Therefore, attempts should be made to collect sufficient information by using limited hardware devices. In addition to the hardware equipment used to collect environmental information, software models must be developed for each application scene for analyzing the collected information. Such models are usually designed according to certain environmental conditions and cannot be updated automatically with changes in the environment, which decreases the flexibility and life cycle of the system on which these models are installed. Therefore, in this study, we developed a Quadro-W (QW) learning method to predict human behavior. QW encompasses humans (who), objects (what), locations (where), and time (when). This system obtained QW information only from the data collected by cameras and did not use additional sensors. This study constructed a behavior prediction model on the basis of the obtained QW information. The developed model can not only make predictions based on the initial environment but also update itself with changes in the environment to increase the system flexibility and life cycle.