Model-free reinforcement learning algorithms, such as Q-learning, perform poorly in the early stages of learning in noisy environments, because much effort is spent unlearning biased estimates of the state-action value function. The bias results from selecting, among several noisy estimates, the apparent optimum, which may actually be suboptimal. We propose G-learning, a new off-policy learning algorithm that regularizes the value estimates by penalizing deterministic policies in the beginning of the learning process. We show that this method reduces the bias of the value-function estimation, leading to faster convergence to the optimal value and the optimal policy. Moreover, G-learning enables the natural incorporation of prior domain knowledge, when available. The stochastic nature of G-learning also makes it avoid some exploration costs, a property usually attributed only to on-policy algorithms. We illustrate these ideas in several examples, where G-learning results in significant improvements of the convergence rate and the cost of the learning process.
Attempts to prove the usefulness of geriatric consultation teams (GCT) in acute-care settings have been inconclusive. We have completed a prospective, controlled trial of a GCT in an acute-care setting, aiming our interventions at a specific subgroup of elderly patients. One hundred and thirty-two out of 352 (37.5%) patients met the inclusion criteria with 66 each being assigned to the intervention and the control groups. There were no significant differences in baseline characteristics between the two groups. Patients in the intervention group received follow-up after discharge from hospital by the geriatric service. We found that the intervention was associated with improved 6-month survival (p less than 0.01), improved Barthel Index at 1 year (p less than 0.01), and a trend towards decreased reliance on institutional care (hospital or nursing home) during the year of follow-up. The benefits occurred principally in patients who were discharged to a nursing home. Our findings support the utility of GCT and highlight the importance of focusing the intervention and providing follow-up after discharge from hospital.
Single-domain particles of Fe3O4 have been observed to exhibit superparamagnetic behavior with the Mössbauer effect. Using a simple theory for the spin relaxation, it was possible to make an approximate curve fit of the spectra, taking into account the particle-size distribution of the samples. For a sample prepared by a grinding method, the values obtained by the fitting procedure were Ku=(6.0±1.0)×104 erg/cm3 for the energy barrier and τ0=(9.5±1.5) × 10−10 sec for the frequency factor. The value of Ku was interpreted as being due to shape anisotropy; analysis of the hyperfine spectra of both bulk and fine-particle Fe3O4 at T=4.2°K supported this view. The flipping frequencies of the particle moments were altered by the application of a magnetic field, so that the average internal field in a relaxed spectrum could be determined. Other samples of Fe3O4, prepared by a precipitation method, gave the values Ku=6.0×104 and 8.0×104 erg/cm3. The latter value could be partially accounted for by the existence of a small oxygen excess in those particles prepared at temperatures close to 100°C.
Rationale: Enhancing non-CFTR (cystic fibrosis transmembrane conductance regulator)-mediated anion secretion is an attractive therapeutic approach for the treatment of cystic fibrosis (CF) and other mucoobstructive diseases. Objectives: To determine the effects of TMEM16A potentiation on epithelial fluid secretion and mucociliary clearance. Methods: The effects of a novel low-molecular-weight TMEM16A potentiator (ETX001) were evaluated in human cell and animal models of airway epithelial function and mucus transport. Measurements and Main Results: Potentiating the activity of TMEM16A with ETX001 increased the Ca 21-activated Cl 2 channel activity and anion secretion in human bronchial epithelial (HBE) cells from patients with CF without impacting calcium signaling. ETX001 rapidly increased fluid secretion and airway surface liquid height in CF-HBE cells under both static conditions and conditions designed to mimic the shear stress associated with tidal breathing. In ovine models of mucus clearance (tracheal mucus velocity and mucociliary clearance), inhaled ETX001 was able to accelerate clearance both when CFTR function was reduced by administration of a pharmacological blocker and when CFTR was fully functional. Conclusions: Enhancing the activity of TMEM16A increases epithelial fluid secretion and enhances mucus clearance independent of CFTR function. TMEM16A potentiation is a novel approach for the treatment of patients with CF and non-CF mucoobstructive diseases.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.