A key element in the solution of reinforcement learning problems is the value function. The purpose of this function is to measure the long-term utility or value of any given state. The function is important because an agent can use this measure to decide what to do next. A common problem in reinforcement learning when applied to systems having continuous states and action spaces is that the value function must operate with a domain consisting of real-valued variables, which means that it should be able to represent the value of infinitely many state and action pairs. For this reason, function approximators are used to represent the value function when a close-form solution of the optimal policy is not available. In this article, we extend a previously proposed reinforcement learning algorithm so that it can be used with function approximators that generalize the value of individual experiences across both state and action spaces. In particular, we discuss the benefits of using sparse coarse-coded function approximators to represent value functions and describe in detail three implementations: cerebellar model articulation controllers, instancebased, and case-based. Additionally, we discuss how function approximators having different degrees of resolution in different regions of the state and action spaces may influence the performance and learning efficiency of the agent. We propose a simple and modular technique that can be used to implement function approximators with nonuniform degrees of resolution so that the value function can be represented with higher accuracy in important regions of the state and action spaces. We performed extensive experiments in the double-integrator and pendulum swing-up systems to demonstrate the proposed ideas.'

No abstract

BackgroundNovel methods of promoting self-monitoring and social support are needed to ensure long-term maintenance of behavior change. In this paper, we directly investigate the effects of group support in an exercise and nutrition program delivered by an mHealth application called Fittle.ObjectiveOur first specific study aim was to explore whether social support improved adherence in wellness programs. Our second specific study aim was to assess whether media types (ePaper vs mobile) were associated with different levels of compliance and adherence to wellness programs. The third aim was to assess whether the use of an mHealth application led to positive changes to participants’ eating behavior, physical activity, and stress level, compared to traditional paper-based programs.MethodsA 2 × 2 (eg, Media: Mobile vs ePaper × Group Type: Team vs Solo) factorial design feasibility study was conducted. A sample of 124 volunteers who were interested in improving eating behavior, increasing physical activity, or reducing stress participated in this study. The study duration was 8 weeks. All groups were self-directed with no ongoing human input from the research team.ResultsParticipants in ePaper conditions had higher attrition rates compared to participants in Mobile conditions, χ3 2=9.96, P=.02 (N=124). Participants in Mobile conditions reported their compliance with a much higher frequency closer to the time of challenge activity completion (2-sample Kolmogorov-Smirnov test comparing distributions was highly significant—KS=0.33, P<.001 [N=63]). Participants in ePaper conditions had a much higher frequency of guessing while reporting as compared with those in Mobile conditions—χ1 2=25.25, P<.001 (N=63). Together, these findings suggest that the mobile app allowed a more accurate method to report and track health behaviors over a longer period than traditional ePaper-based diaries or log books. There was a significant difference in the overall compliance score for Mobile-Solo (Mean [SD] 0.30 [0.39]) and Mobile-Team (Mean [SD] 0.49 [0.35]) conditions (t 50.82=1.94, P=.05). This suggests that working in a team increased participants’ overall compliance within Fittle. Survival analysis showed that participants assigned to Team conditions are 66% more likely to engage longer with mHealth app-based intervention than those assigned to the Solo condition. Overall, participants across all groups reported some positive changes in eating behavior, physical activity, and stress level; however, participants in the Mobile-Solo condition reported higher perceived stress levels at the end of the study.ConclusionsThe team-based Fittle app is an acceptable and feasible wellness behavior change intervention and a full randomized controlled trial to investigate the efficacy of such an intervention is warranted.

No abstract

A growing research community is working toward employing drama management components in storybased games. These components gently guide the story toward a narrative arc that improves the player's gaming experience. In this article we evaluate a novel drama management approach deployed in an interactive fiction game called Anchorhead. This approach uses player's feedback as the basis for guiding the personalization of the interaction. The results indicate that adding our Case-based Drama manaGer (C-DraGer) to the game guides the players through the interaction and provides a better overall player experience. Unlike previous approaches to drama management, this article focuses on exhibiting the success of our approach by evaluating results using human players in a real game implementation. Based on this work, we report several insights on drama management which were possible only due to an evaluation with real players.