Toward Affordable and Practical Home Context Recognition: –Framework and Implementation with Image-based Cognitive API–

2020

Self Cite

Cognitive Application Program Interface (API) is an API of emerging artificial intelligence (AI)-based cloud services, which extracts various contextual information from non-numerical multimedia data including image and audio. Our interest is to apply image-based cognitive APIs to implement flexible and efficient context sensing services in a smart home. In the existing approach with machine learning by us, with the complexity of recognition object and the number of the defined contexts increases by users, it still requires directly manually labeling a moderate scale of data for training and continually try to calling multiple cognitive APIs for feature extraction. In this paper, we propose a novel method that uses a small scale of labeled data to evaluate the capability of cognitive APIs in advance, before training features of the APIs with machine learning, for the flexible and efficient home context sensing. In the proposed method, we exploit document similarity measures and the concepts (i.e., internal cohesion and external isolation) integrate into clustering results, to see how the capability of different cognitive APIs for recognizing each context. By selecting the cognitive APIs that relatively adapt to the defined contexts and data based on the evaluation results, we have achieved the flexible integration and efficient process of cognitive APIs for home context sensing.

Section: Proposed Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Toward Flexible and Efficient Home Context Sensing: Capability Evaluation and Verification of Image-Based Cognitive APIs

2020

Self Cite

“…Constructing a single classifier model is a basic and essential part of realizing fine-grained home context recognition. Unlike naive deep learning, we previously dedicated the features of images extracted from a single cognitive API, to apply to light-weight supervised machine learning [29].…”

Section: Preliminary Studymentioning

confidence: 99%

“…For this purpose, we are currently investigating techniques that integrate inexpensive camera devices, multiple image-based cognitive APIs, and light-weight machine learning. We previously encoded the tags of a single API to document vectors, then applied them into machine learning for the model construction [29]. However, we found that the accuracy significantly decreased for contexts with multiple people (e.g., "General meeting", "Dining together", "Play games").…”

mentioning

confidence: 99%

Integrating Multiple Models Using Image-as-Documents Approach for Recognizing Fine-Grained Home Contexts

2020

Self Cite

To implement fine-grained context recognition that is accurate and affordable for general households, we present a novel technique that integrates multiple image-based cognitive APIs and light-weight machine learning. Our key idea is to regard every image as a document by exploiting "tags" derived by multiple APIs. The aim of this paper is to compare API-based models' performance and improve the recognition accuracy by preserving the affordability for general households. We present a novel method for further improving the recognition accuracy based on multiple cognitive APIs and four modules, fork integration, majority voting, score voting, and range voting.Our interest is to use little image data, applying cognitive services to implement affordable context sensing that can adapt to custom contexts in every single house. The cognitive service is a cloud service with cognitive computing functions that provide the capability to understand multimedia data (i.e., vision (object recognition) [19], speech recognition [20], natural language processing [21], etc.), based on sophisticated machine-learning algorithms powered by the offered big data and large-scale computing. They are offered by cloud companies, but the data processing algorithms behind them are not public. They are also widely used in various fields of research, such as modern knowledge management solutions [22] and criminal detection and recognition [23]. The cognitive API is an application program interface via the HTTP/REST protocol [24], with which developers can easily integrate powerful recognition features in their own applications. An image-based cognitive API receives an image from an external application, i.e., extracts specific information from the image, and returns the information in JavaScript Object Notation (JSON) [25] format from the cloud server rather than the local. The information usually contains a set of words called "tags", representing objects and concepts that the API has recognized in the given image. Examples of tags from the API are: [Living, room, indoors, classroom, basement, supporting structure]. The information of interest and the way of recognizing the image vary among individual cognitive services. Related work uses image tagging technology with deep learning, as in [26][27][28], but the implementation is more complex. In our future realistic implementation, for security and privacy of the users the images are not saved after sending the APIs over.The main contribution of this paper is to present a novel method which is not only affordable but also has higher accuracy in recognizing fine-grained home contexts. For this purpose, we are currently investigating techniques that integrate inexpensive camera devices, multiple image-based cognitive APIs, and light-weight machine learning. We previously encoded the tags of a single API to document vectors, then applied them into machine learning for the model construction [29]. However, we found that the accuracy significantly decreased for contexts with multiple people (e.g., "...

“…(2) Real-time analysis using a large number of computing resources required is unrealistic. (3) The possibility of leakage of information cannot be ignored when uploading data through the network as demonstrated in [ 34 , 35 , 36 ]. In this section, we introduce some related works from recent years.…”

Section: Related Workmentioning

confidence: 99%

Nonintrusive Fine-Grained Home Care Monitoring: Characterizing Quality of In-Home Postural Changes Using Bone-Based Human Sensing

2020

Self Cite

In contrast to the physical activities of able-bodied people at home, most people who require long-term specific care (e.g., bedridden patients and patients who have difficulty walking) usually show more low-intensity slow physical activities with postural changes. Although the existing devices can detect data such as heart rate and the number of steps, they have been increasing the physical burden relying on long-term wearing. The purpose of this paper is to realize a noninvasive fine-grained home care monitoring system that is sustainable for people requiring special care. In the proposed method, we present a novel technique that integrates inexpensive camera devices and bone-based human sensing technologies to characterize the quality of in-home postural changes. We realize a local process in feature data acquisition once per second, which extends from a computer browser to Raspberry Pi. Our key idea is to regard the changes of the bounding box output by standalone pose estimation models in the shape and distance as the quality of the pose conversion, body movement, and positional changes. Furthermore, we use multiple servers to realize distributed processing that uploads data to implement home monitoring as a web service. Based on the experimental results, we conveyed our findings and advice to the subject that include where the daily living habits and the irregularity of home care timings needed improvement.