Publications supported by lorAI
-
Towards Trustworthy Multi-stakeholder Recommender Systems
Recommender systems now have to balance the needs of dozens of stakeholders, each often juggling multiple, sometimes conflicting objectives at once. While multi-objective and multi-stakeholder RS have been studied, trustworthiness in these systems has gotten far less attention despite becoming an emerging standard. This paper lays out the open challenges and key questions that must be addressed to move toward trustworthy multi-stakeholder recommender systems.
-
dfkinit2b at CheckThat! 2025: Leveraging LLMs and Ensemble of Methods for Multilingual Claim Normalization
The CLEF 2025 CheckThat! shared task tackles transforming noisy social media posts into normalized, check-worthy claims across multiple languages. This paper tests parameter-efficient fine-tuning, LLM prompting, and method ensembles in both monolingual and zero-shot settings. The approaches placed first in 6 of 13 languages monolingually and topped all seven languages in the zero-shot setting, showing strong generalization to unseen languages.
-
mdok of KInIT: Robustly Fine-tuned LLM for Binary and Multiclass AI-Generated Text Detection
LLMs can generate text in multiple languages that humans often can’t distinguish from human-written content, raising risks like plagiarism and disinformation. This work presents mdok, an approach for robustly detecting AI-generated text by fine-tuning smaller LLMs for classification, tested on out-of-distribution data. Applied to both subtasks of Voight-Kampff Generative AI Detection 2025, mdok ranked first in both binary detection and multiclass classification of human-AI collaboration.
-
The Brittle Compass: Navigating LLM Prompt Sensitivity in Slovak Migration Media Discourse
This case study looks at migration-related tasks in Slovak, including topic relevance, geographical relevance classification, and location term extraction. Native Slovak prompts bring modest, task-dependent gains, while larger models show far more robustness to prompt variations than smaller ones. Instructions, whether system or task-level, turn out to matter more than examples, with task-specific effects outweighing overall language effects.
-
o-MEGA: Optimized Methods for Explanation Generation and Analysis
Hyperparameter optimization tool built to automatically identify the most effective explainable AI methods and configurations within semantic matching. It’s evaluated on a post-claim matching pipeline using social media posts paired with refuting claims, systematically testing different explainability methods and their settings. The results show improved transparency in automated fact-checking systems, making claim-matching models more interpretable for misinformation detection.
-
AI Research is not Magic, it has to be Reproducible and Responsible: Challenges in the AI field from the Perspective of its PhD Students
A survey of 28 AI doctoral candidates from 13 European countries surfaces challenges around findability of AI resources, experiment replication, and lack of trustworthiness and interdisciplinarity. Early-stage researchers tend to share resources but often lack motivation or knowledge for proper data curation, ethical assessment, and cross-domain collaboration. The paper proposes both social fixes, like wider adoption of reproducibility initiatives, and technical ones, like better versioning tools and shared infrastructure for AI resources.
-
Multilingual Political Views of Large Language Models: Identification and Steering
Seven open-source instruction-tuned LLMs, including LLaMA-3.1, Qwen-3, and Aya-Expanse, are evaluated across 14 languages using the Political Compass Test with 11 paraphrases per statement for robustness. Larger models consistently lean libertarian-left, with notable variation across languages and model families. A simple activation intervention technique is also shown to reliably steer model responses toward alternative ideological positions across languages.
-
Between Principle and Practice: Evaluating the EU AI Act Through the Lens of Digital Humanism
The EU AI Act is analyzed through Erich Prem’s Digital Humanism principles, covering the co-evolution of technology and humanity, protective responsibility, democratic reinforcement, and the fundamental differences between humans and machines. The analysis finds the Act incorporates some of these principles, notably through its focus on fundamental rights, but only partially fulfills their broader vision. A deeper alignment with digital humanism, the paper argues, would strengthen the Act’s ability to genuinely prioritize humans and societal wellbeing.
-
Beyond the Checkbox: Strengthening DSA Compliance Through Social Media Algorithmic Auditing
The Digital Services Act requires platforms to undergo independent audits verifying compliance with rules on algorithmic transparency, user protection, and privacy. This paper examines selected audit reports across three provisions, restrictions on profiling minors, recommender system transparency, and limits on targeted advertising with sensitive data, finding significant inconsistencies and shallow technical depth. To strengthen these assessments, the authors propose algorithmic auditing that simulates user behaviour and analyses platform responses directly.
-
PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
Parameter-efficient fine-tuning methods reduce the computational and environmental costs of LLMs by limiting the number of trainable parameters, but evaluations of these methods remain limited and difficult to reproduce. PEFT-Bench addresses this with a unified benchmark tested across 27 NLP datasets and 7 PEFT methods on autoregressive LLMs. It also introduces the PEFT Soft Cost Penalties metric, accounting for trainable parameters, inference speed, and training memory usage together.
-
RoSE: Round-robin Synthetic Data Evaluation for Selecting LLM Generators without Human Test Sets
Picking the best LLM to generate synthetic training data is hard without costly human annotations, since intrinsic metrics correlate poorly with actual downstream performance. RoSE addresses this by training a small model on one LLM’s synthetic outputs, then testing it on outputs from all other candidate LLMs, averaging the results into a proxy score. Across six LLMs, eleven languages, and three tasks, RoSE outperforms other intrinsic heuristics, landing within 0.76 points of the optimal generator and uniquely correlating positively with human-labelled test performance.
-
Riding the Carousel: The First Extensive Eye Tracking Analysis of Browsing Behavior in Carousel Recommenders
Eye tracking reveals how users actually browse carousel recommenders, the multi-list, swipeable interfaces now standard across streaming and e-commerce platforms. This study provides the first extensive analysis of browsing patterns, examining where users start looking, how their gaze transitions between and within carousels, and how genre preferences shape those transitions. Findings point to concrete design takeaways, especially reordering ranked items to match better how users scan after swiping.
-
PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models
Newly introduced PEFT methods are often hard to replicate, deploy, or compare against each other. PEFT-Factory addresses this with a unified, modular framework built on LLaMA-Factory, natively supporting 19 PEFT methods across 27 datasets and 12 tasks. It provides both standard and PEFT-specific evaluation metrics, giving a controlled, stable environment for benchmarking and improving replicability across methods.
-
mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection
SemEval-2026 Task 9 covers multilingual, multicultural, and multievent polarization detection along three axes: detection, type, and manifestation. Since online polarization often precedes hate speech and social fragmentation, catching it early matters for safer online spaces. This approach fine-tunes mid-size LLMs for sequence classification using QLoRA, augmenting the 22-language training data with anonymized, lower-cased, upper-cased, and homoglyphied variants to improve robustness.
-
mcdok at SemEval-2026 Task 13: Finetuning LLMs for Detection of Machine-Generated Code
Machine-generated code detection across multiple programming languages gets tackled from several angles, including binary detection, generator attribution, and adversarially disguised code. This work adapts the existing mdok approach, originally built for text detection, to code understanding with better-suited base models. The submitted systems prove competitive across all three subtasks, though gaps remain versus top performers, leaving room for improvement.
-
BLUFF: Benchmarking the Detection of False and Synthetic Content across 58 Low-Resource Languages
Detection of false and machine-generated content remains focused on English and a few high-resource languages, leaving many communities without reliable tools. BLUFF is a multilingual benchmark with over 202K samples in 79 languages, combining human-written and LLM-generated texts. State-of-the-art detectors perform up to 25.3% worse (F1) on low-resource languages.
-
Want Better Synthetic Data? Steer It: Activation Steering for Low Resource Language Generation
Few-shot prompting is the usual way to generate synthetic data for low-resource languages, but it raises inference costs and can reduce diversity. This work explores activation steering as an alternative, tested across four LLMs and 11 diverse languages. Steering on early layers consistently improves data diversity and often boosts downstream classifier performance, especially for low-resource languages.
-
sk-bench: A Native-First Benchmark for Evaluating Large Language Models in Slovak
Multilingual LLM benchmarks either omit Slovak or cover it only through machine translation. sk-bench is a native-first benchmark with 30 datasets across ten skill categories, used to evaluate 55 open- and closed-weights models. The best open model trails proprietary APIs by 12.6 points, and test-time reasoning improves scores by 8.5 to 12.5 points for models of 9B and above.
-
Interpretable Predictability-Based AI Text Detection: A Replication Study
This paper replicates and extends a system from the AuTexTification 2023 shared task on attributing machine-generated texts. Exact replication proved impossible due to differences in data splits, model availability, and implementation details, so newer multilingual models and 26 stylometric features were added. These extra features improve performance in both subtasks and both languages, and one shared multilingual configuration matches or beats language-specific models.
-
Are XAI Metrics Meaningful Proxies for Human Judgment? Evaluating Explanation Quality in Claim Matching
AutoXAI frameworks choose explainability methods using metrics such as faithfulness and plausibility, but it is unclear whether these match how people judge explanation quality. A large-scale evaluation of 11 XAI methods, paired with a user study on claim matching, shows that methods favored by these metrics are also rated most helpful by users, while deprioritized ones are seen as misleading. Simple post-processing of explanations can further improve perceived usefulness without hurting faithfulness.
Other publications in collaboration with lorAI partners
-
Soft Language Prompts for Language Transfer
Cross-lingual knowledge transfer between high- and low-resource languages gets tested across 16 languages, including 10 mid- and low-resource ones, using combinations of language adapters, task adapters, and soft prompts. The work introduces “soft language prompts,” the first known use of soft prompts for language transfer. Pairing a soft language prompt with a task adapter outperforms most other combinations, contradicting earlier claims that language and task adapters work best together.
-
Multimodal and Multilingual Fact-Checked Article Retrieval
M3-Check is introduced as the first fact-check retrieval dataset combining multilingual text and images from social media posts with fact-check articles from diverse, credible sources. Alongside it, FACTOR, a two-tower Transformer architecture with cross-tower parameter sharing and modality-aligned weight initialization, outperforms zero-shot baselines and vanilla Transformers by 17%. Modality ablations further show that multilingual encoders like multi-E5 add another 13% in performance without needing English translations.
-
Task Prompt Vectors: Effective Initialization Through Multi-task Soft Prompt Transfer
Task Prompt Vectors extend the idea of task vectors, computed on full model weights, to soft prompts, taking the element-wise difference between a tuned prompt and its random initialization. Across 19 datasets, these vectors effectively initialize prompt tuning on similar tasks in low-resource settings, and hold up across 3 different language model architectures regardless of random initialization. This independence from initialization means task prompt vectors from different tasks can be combined arithmetically, offering a competitive and cheaper alternative to state-of-the-art multi-task prompt tuning methods.
-
Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages
Three adapter architectures, Sequential Bottleneck, Invertible Bottleneck, and Low-Rank Adaptation, are tested for adapting small multilingual models like mBERT and XLM-R to low-resource languages, using small amounts of text and knowledge graph data. Sequential Bottleneck adapters work best for language modeling, while Invertible Bottleneck adapters edge ahead on downstream tasks like classification and NER. Adapter-based methods match or beat full fine-tuning with far fewer parameters, and even outperform massive LLMs like LLaMA-3 and GPT-4 on these languages.
-
Automatic Fact-checking in English and Telugu
Manually verifying factual claims is slow and resource-heavy, and this work tests how well large language models can classify claim veracity and generate justifications in English and Telugu. A bilingual English-Telugu dataset is created as part of the effort. Different LLM-based veracity classification approaches are then benchmarked against each other on this dataset.
-
A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages
Generation strategies like demonstrations, label-based summaries, and self-revision are systematically compared for creating synthetic training data in low-resource languages. Testing spans 11 typologically diverse languages, three NLP tasks, and four open-source LLMs, comparing downstream performance on generated versus gold-standard data. Combining target-language demonstrations with LLM-based revisions works best, narrowing the gap to real data to as little as 5%, and smart prompting also reduces the advantage of larger LLMs.
-
Large Language Models for Multilingual Previously Fact-Checked Claim Detection
Detecting previously fact-checked claims across languages helps fact-checkers avoid duplicating work as misinformation spreads across borders. This paper presents the first comprehensive evaluation of LLMs for this task, testing seven models across 20 languages in monolingual and cross-lingual settings. LLMs perform well on high-resource languages but struggle with low-resource ones, though translating texts into English helps close that gap.
-
SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval
SemEval 2025 hosted a shared task on multilingual claim retrieval, matching newly encountered social media claims to fact-checked ones across languages, split into monolingual and crosslingual subtracks. The task drew 179 registered participants and 52 test submissions, with 23 of 31 teams submitting system papers. This paper reports the best-performing systems and most effective approaches across both subtracks, offering insights for future multilingual fact-checking research.
-
False Alarm or Real Threat? Trends in GenAI-Mediated Disinformation
This chapter weighs whether deepfakes and generative AI represent a genuine disinformation threat or an overblown concern, as increasingly realistic synthetic media raises the stakes for information integrity and public trust. It examines recent trends in GenAI content across platforms, looking at its disruptive potential in public debate, election campaigns, and conflicts. The goal is to map this complex landscape and highlight the need for effective countermeasures, while acknowledging GenAI’s potential for positive use too.
-
Assessing Web Search Credibility and Response Groundedness in Chat Assistants
Chat assistants with web search can amplify misinformation if they cite low-credibility sources, so this paper introduces a methodology for evaluating source credibility and response groundedness. Using 100 claims across five misinformation-prone topics, GPT-4o, GPT-5, Perplexity, and Qwen Chat are compared. Perplexity comes out with the highest source credibility, while GPT-4o cites non-credible sources more often on sensitive topics, making this the first systematic comparison of these assistants’ fact-checking behaviour.
Deliverables
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D1.1 | Project Governance & Procedures Manual & Management Plan | KInIT | M6 – Aug 2025 | No |
| D1.2 | Data management plan | KInIT | M6 – Aug 2025 | Yes |
| D1.6 | Implementation report 1 | KInIT | M18 – Aug 2026 | No |
| D1.3 | Data management plan mid-term update | KInIT | M36 – Feb 2028 | Yes |
| D1.7 | Implementation report 2 | KInIT | M36 – Feb 2028 | No |
| D1.5 | CoE knowledge base | KInIT | M54 – Aug 2029 | No |
| D1.8 | Implementation report 3 | KInIT | M54 – Aug 2029 | No |
| D1.4 | Data management plan final update | KInIT | M72 – Feb 2031 | Yes |
| D1.9 | Implementation report 4 | KInIT | M72 – Feb 2031 | No |
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D2.1 | R&I and sustainability strategy | TCD/ADAPT | M18 – Aug 2026 | No |
| D2.2 | Infrastructure enhancement plan | KInIT | M36 – Feb 2028 | No |
| D2.3 | Upgraded Grant Management Office | KInIT | M48 – Feb 2029 | No |
| D2.5 | Blueprints of CoE processes | KInIT | M54 – Aug 2029 | No |
| D2.4 | Post-project transition plan and strategy | KInIT | M72 – Feb 2031 | No |
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D3.1 | Talent acquisition kit | KInIT | M18 – Aug 2026 | No |
| D3.2 | Career development kit | KInIT | M36 – Feb 2028 | No |
| D3.3 | ESR engagement event | TCD/ADAPT | M54 – Aug 2029 | Yes |
| D3.5 | ERC systematic candidate identification and attraction processes | TCD/ADAPT | M54 – Aug 2029 | No |
| D3.4 | Self-assessment tool for identifying strengths and weaknesses of employees | KInIT | M72 – Feb 2031 | No |
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D4.1 | Training plan and strategy | CERTH | M18 – Aug 2026 | No |
| D4.2 | Mobility plan and strategy | DFKI | M18 – Aug 2026 | No |
| D4.4 | ESR desk and processes | TCD/ADAPT | M54 – Aug 2029 | No |
| D4.3 | Catalogue of training assets and services | KInIT | M72 – Feb 2031 | Yes |
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D5.1 | Guidelines for assessment of LRAI models robustness | TCD/ADAPT | M18 – Aug 2026 | Yes |
| D5.2 | LRAI robustness assessment framework | TCD/ADAPT | M36 – Feb 2028 | No |
| D5.3 | Pilot Research Project results. LRAI robustness enhancement methods | CERTH | M60 – Feb 2030 | Yes |
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D6.1 | Catalogue of assessment services | KInIT | M36 – Feb 2028 | No |
| D6.3 | Technology transfer desk and processes | DFKI | M48 – Feb 2029 | No |
| D6.2 | Catalogue of assessment services final update | KInIT | M72 – Feb 2031 | No |
| D6.4 | Catalogue of flagship assets and services | KInIT | M72 – Feb 2031 | No |
| No. | Deliverable | Lead | Due | Public document |
|---|---|---|---|---|
| D7.4 | Project website | KInIT | M3 – May 2025 | This website |
| D7.1 | Plan for the dissemination and exploitation including communication activities | KInIT | M6 – Aug 2025 | Yes |
| D7.2 | Plan for the dissemination and exploitation including communication activities mid-term update | KInIT | M36 – Feb 2028 | Yes |
| D7.5 | Networking strategy | KInIT | M36 – Feb 2028 | Yes |
| D7.6 | List of research groups (inc. diaspora) suitable for collaboration | KInIT | M54 – Aug 2029 | No |
| D7.3 | Plan for the dissemination and exploitation including communication activities final update | KInIT | M72 – Feb 2031 | Yes |
| D7.7 | Policy advising team and processes | KInIT | M72 – Feb 2031 | No |
Datasets
-
RecGaze Click Feedback Dataset
The RecGaze Click Feedback Dataset is the largest publicly available carousel feedback dataset specifically designed for click modeling. It contains 2,387 sessions and clicks (i.e. users only selected/clicked one movie), 1,307 unique movie items, and 87 unique users.
-
BLUFF
BLUFF is a comprehensive benchmark for detecting false and synthetic content, spanning 79 languages with over 202K samples, combining humanwritten fact-checked content and LLM-generated content.
-
PerQ
A dataset for evaluation of platform-adaptation (Twitter/X, Telegram, and Signal) capabilities of 6 LLMs (smaller and bigger versions of the 3 families: Gemma-3, Qwen3, Llama-3) for 7 languages (English, German, French, Italian, Slovak, Russian, and Hungarian). It has been used for training of resource-efficient metric approximating the majority-voted judgement of 3 LLMs.