Disentangling conversations mixed together in a single stream of messages is a difficult task, made harder by the lack of large manually annotated datasets. We created a new dataset of 77,563 messages manually annotated with reply-structure graphs that both disentangle conversations and define internal conversation structure. Our dataset is 16 times larger than all previously released datasets combined, the first to include adjudication of annotation disagreements, and the first to include context. We use our data to re-examine prior work, in particular, finding that 80% of conversations in a widely used dialogue corpus are either missing messages or contain extra messages. Our manually-annotated data presents an opportunity to develop robust data-driven methods for conversation disentanglement, which will help advance dialogue research.

To cope with large data sets, distributed data stores partition their data across servers. However, real-world systems usually do not provide useful transactional semantics for operations accessing multiple partitions due to the delays involved in achieving multi-partition consistency. Read Atomic Multi-Partition (RAMP) transactions have recently been proposed as efficient lightweight multi-partition transactions that guarantee read atomicity: either all updates or no updates of a transaction are visible to other transactions. In this paper we formalize RAMP transactions in rewriting logic and perform model checking verification of key properties using the Maude tool. In particular, we develop detailed formal models-and formally analyze-a number of extensions and optimizations of RAMP that are only briefly mentioned by the RAMP developers.

In a dialog, there can be multiple valid next utterances at any point. The present end-toend neural methods for dialog do not take this into account. They learn with the assumption that at any time there is only one correct next utterance. In this work, we focus on this problem in the goal-oriented dialog setting where there are different paths to reach a goal. We propose a new method, that uses a combination of supervised learning and reinforcement learning approaches to address this issue. We also propose a new and more effective testbed, permuted-bAbI dialog tasks 1 by introducing multiple valid next utterances to the original-bAbI dialog tasks, which allows evaluation of goal-oriented dialog systems in a more realistic setting. We show that there is a significant drop in performance of existing end-toend neural methods from 81.5% per-dialog accuracy on original-bAbI dialog tasks to 30.3% on permuted-bAbI dialog tasks. We also show that our proposed method improves the performance and achieves 47.3% per-dialog accuracy on permuted-bAbI dialog tasks. * Equal Contribution 1 permuted-bAbI-dialog-taskshttps://github. com/IBM/permuted-bAbI-dialog-tasks arXiv:1808.09996v1 [cs.CL]

The promise of high scalability and availability has prompted many companies to replace traditional relational database management systems (RDBMS) with NoSQL key-value stores. This comes at the cost of relaxed consistency guarantees: key-value stores only guarantee eventual consistency in principle. In practice, however, many key-value stores seem to offer stronger consistency. Quantifying how well consistency properties are met is a non-trivial problem. We address this problem by formally modeling key-value stores as probabilistic systems and quantitatively analyzing their consistency properties by both statistical model checking and implementation evaluation. We present for the first time a formal probabilistic model of Apache Cassandra, a popular NoSQL key-value store, and quantify how much Cassandra achieves various consistency guarantees under various conditions. To validate our model, we evaluate multiple consistency properties using two methods and compare them against each other. The two methods are: (1) an implementationbased evaluation of the source code; and (2) a statistical model checking analysis of our probabilistic model.

Abstract. In this paper we explore and extend the design space of the recent RAMP (Read Atomic Multi-Partition) transaction system for large-scale partitioned data stores. Arriving at a mature distributed system design through implementation and experimental validation is a labor-intensive task, so that only a limited number of design alternatives can be explored in practice. The developers of RAMP did implement and validate three design alternatives for RAMP, and sketched three additional designs. This work addresses two questions: (1) How can the design space of a distributed transaction system such as RAMP be explored with modest effort, so that substantial knowledge about design alternatives can be gained before designs are implemented? and (2) How realistic and informative are the results of such design explorations? We answer the first question by: (i) formally modeling eight RAMP-like designs (five by the RAMP developers and three of our own) in Maude as probabilistic rewrite theories, and (ii) using statistical model checking of those models to analyze key performance metrics such as throughput, average latency, and degrees of strong consistency and read atomicity. We answer the second question by showing that our quantitative analyses: (i) are consistent with the experimental results obtained by the RAMP developers for their implemented designs; (ii) confirm the conjectures made by the RAMP developers for their other three unimplemented designs; and (iii) uncover some promising new designs that seem attractive for some applications.

Neural end-to-end goal-oriented dialog systems showed promise to reduce the workload of human agents for customer service, as well as reduce wait time for users. However, their inability to handle new user behavior at deployment has limited their usage in real world. In this work, we propose an end-to-end trainable method for neural goal-oriented dialog systems which handles new user behaviors at deployment by transferring the dialog to a human agent intelligently. The proposed method has three goals: 1) maximize user's task success by transferring to human agents, 2) minimize the load on the human agents by transferring to them only when it is essential and 3) learn online from the human agent's responses to reduce human agents load further. We evaluate our proposed method on a modified-bAbI dialog task that simulates the scenario of new user behaviors occurring at test time. Experimental results show that our proposed method is effective in achieving the desired goals.

A frequent pattern in customer care conversations is the agents responding with appropriate webpage URLs that address users' needs. We study the task of predicting the documents that customer care agents can use to facilitate users' needs. We also introduce a new public dataset 1 which supports the aforementioned problem. Using this dataset and two others, we investigate state-of-the-art deep learning (DL) and information retrieval (IR) models for the task. We also analyze the practicality of such systems in terms of inference time complexity. Our results show that an hybrid IR+DL approach provides the best of both worlds.

No abstract