In some memory-constrained settings like IoT devices and over-the-network data pipelines, it can be advantageous to have smaller contextual embeddings. We investigate the efficacy of projecting contextual embedding data (BERT) onto a manifold, and using nonlinear dimensionality reduction techniques to compress these embeddings. In particular, we propose a novel post-processing approach, applying a combination of Isomap and PCA. We find that the geodesic distance estimations, estimates of the shortest path on a Riemannian manifold, from Isomap's k-Nearest Neighbors graph bolstered the performance of the compressed embeddings to be comparable to the original BERT embeddings. On one dataset, we find that despite a 12-fold dimensionality reduction, the compressed embeddings performed within 0.1% of the original BERT embeddings on a downstream classification task. In addition, we find that this approach works particularly well on tasks reliant on syntactic data, when compared with linear dimensionality reduction. These results show promise for a novel geometric approach to achieve lower dimensional text embeddings from existing transformers and pave the way for dataspecific and application-specific embedding compressions.
Inspired by Gibson's notion of object affordances in human vision, we ask the question: how can an agent learn to predict an entire action policy for a novel object or environment given only a single glimpse? To tackle this problem, we introduce the concept of Universal Policy Functions (UPFs) which are state-to-action mappings that generalize not only to new goals but most importantly to novel, unseen environments. Specifically, we consider the problem of efficiently learning such policies for agents with limited computational and communication capacity, constraints that are frequently encountered in edge devices. We propose the Hyper-Universal Policy Approximator (HUPA), a hypernetwork-based model to generate small task-and environment-conditional policy networks from a single image, with good generalization properties. Our results show that HUPAs significantly outperform an embedding-based alternative for generated policies that are size-constrained. Although this work is restricted to a simple map-based navigation task, future work includes applying the principles behind HUPAs to learning more general affordances for objects and environments.
This article is concerned with informality-state relations, subaltern politics and citizenship in the context of the urban redevelopment regime. Based on an empirical study of an NGO (SPARC)-mediated resettlement of Project Affected Persons (PAPs) in Mumbai, it explicates the incomplete ‘civilizing of the political society’ which engenders asymmetrical material and leadership enablement and differential subjectivities at the community levels. The state co-opts SPARC’s institutional framework to mediate resettlement, engender limited traversal from ‘population’ to ‘citizen’, restrict democratic liberation and subject the PAPs to bifold governance against the antagonistic articulations of state-subaltern relations, viz. ‘political society’ and ‘deep democracy’. SPARC’s institutional claims of inclusion and community-centric resettlement, non-confrontational negotiations and politics of patience are materialized through institutional coercion, domesticated confrontations and inadequate compensation, and are augmented by the PAPs’ calculative rationalities, fear of homelessness and anticipation of urban citizenship. Against this backdrop and amid further post-resettlement marginalities that complicate housing-based ‘substantive citizenship’ and ‘political society’-based mediation, this article calls for a re-politicization of the redevelopment discourse to seek alternate possibilities of urban citizenship for the urban subaltern.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.