Optimization of coal blending operations under uncertainty – robust optimization approach

Abstract

Direct preference optimization (DPO) is a successful fine-tuning strategy for aligning large language models with human preferences without the need to train a reward model or employ reinforcement learning. DPO, as originally formulated, relies on binary preference data and fine-tunes a language model to increase the likelihood of a preferred response over a dispreferred response. However, not all preference pairs are equal: while in some cases the preferred response is only slightly better than the dispreferred response, there can be a stronger preference for one response when, for example, the other response includes harmful or toxic content. In this paper, we propose a generalization of DPO, termed DPO with an offset (ODPO), that does not treat every preference pair equally during fine-tuning. Intuitively, ODPO requires the difference between the likelihood of the preferred and dispreferred response to be greater than an offset value. The offset is determined based on the extent to which one response is preferred over another. Our experiments on various tasks suggest that ODPO significantly outperforms DPO in aligning language models, especially when the number of preference pairs is limited.

Dynamic coal blending under sustainability concerns: a case study from the Indian steel industry

“…Jiao et al (2018) use properties of the produced char to investigate the proportion of single coal in blends with three different coals under changeable ranks at 1,1508C. Amini et al (2019) describe coal blending optimization while taking into account the uncertainties involved in coal attributes. They use sampling within a robust optimization framework to guarantee desired levels of coke quality.…”

Kalite ve maliyet perspektiflerinden pirinç alaşımı harmanlama problemi: çok amaçlı bir optimizasyon yaklaşımı

“…The solution of the numerical example has shown that the developed model and solution procedure can be used successfully in real life applications. Ancak problem parametreleri her zaman belirli-kesin değildir; ürüne değer katan girdilere ait bilgiler belirsiz olabilir [15]. Bliss [16], ve Sakalli ve Birgören [11] çalışmalarında belirsizlik içeren harmanlama problemini belirlilik kabulü altında modellemişlerdir.…”

Pozyskiwanie węgla kamiennego z uwzględnieniem wybranych regulacji środowiskowych – studium przypadku polskiego sektora energetycznego

“…Optimizing the supply and use of hard coal in the process of electricity production has already been the subject of many studies. The optimization problem was formulated from the perspective of the coal supplier (Kasana and Kumar 2004;Kozan and liu 2012;Blom et al 2019) the commercial intermediary (Cheng et al 2016;Amini et al 2022), and the consumers. The common point of the published works is an attempt to make management decisions in selling or purchasing raw material, following the adopted optimization criterion, which is most often maximizing profit (of a coal company or an intermediary), minimizing transport costs, or minimizing the cost of coal purchase by the recipient.…”