Leveraging RAG and LLMs for Access Control Policy Extraction From User Stories in Agile Software Development

You’re listening to “Leveraging RAG and LLMs for Access Control Policy Extraction From User Stories in Agile Software Development,” by S. Aboukadri and colleagues. Published in 2025.

Abstract.

Agile development has become increasingly popular among software development teams due to its capacity to deliver and update software rapidly while accommodating evolving requirements. Within this dynamic context, access control policies are critical for ensuring the security of systems by defining who can access specific resources under given conditions. However, identifying and documenting these policies often rely on manual, time-intensive processes prone to errors and oversight. This paper proposes an innovative framework leveraging Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs) to automate the extraction and organization of access control policies from user stories and software documentation. The framework focuses on the early stages of the development lifecycle, capturing access control requirements as expressed in natural language artifacts.

It comprises two core components: 1) a pipeline for extracting and categorizing access control policies, enabling precise mappings between roles, actions, and resources, and 2) an interactive chatbot designed to support Security Operations Center (SOC) analysts in evaluating suspicious access requests by providing contextualized insights into access policies. By integrating advanced natural language processing techniques with retrieval-based augmentation, the framework aims to reinforce access control mechanisms by improving visibility, and providing contextualized insights for security analysts.

Introduction.

Agile software development has become widely popular in the software industry due to its iterative and incremental approach and the emphasis on user participation. However, it has been criticized for its lack of focus on security through-out the development life cycle. Key challenges include the absence of explicit security requirements, inadequate attention to security concerns during development, and the lack of dedicated personnel with specialized knowledge in software security. These gaps highlight the need for improved methods to integrate security into Agile practices, addressing it as a priority rather than an afterthought.. The lack of incentives to prioritize security during the early stages of development, coupled with the absence of integrated

The associate editor coordinating the review of this manuscript and approving it for publication was Sunil Karamchandani.

security practices within Agile frameworks, often results in the neglect of security considerations.

Access control policies are critical components of infor-mation security, defining rules that determine who can access resources, under what conditions, and for what purposes.

In the context of this work, ‘‘resources’’ refer to the elements that users interact with or request access to during normal usage. These may include files, transaction records, system components, microservices, or any domain-specific assets that the application manages. The exact nature of these resources varies depending on the product being developed, but they consistently represent the data or functionality that needs to be protected from unauthorized access while remaining accessible to entitled users.

Access control policies ensure that sensitive data and systems are protected against unauthorized access while allowing legitimate users to perform their tasks efficiently.

The scale and complexity of modern systems, or the lack of access to their source code, can make it challenging for both users and developers to obtain a precise and comprehensive understanding of these policies. This gap can hinder efforts to evaluate, improve, or enforce security measures effectively.

In particular, agile practices often contribute to the introduction and propagation of access control and other security vulnerabilities. Key challenges arise from the nature of agile processes, such as frequent code refactoring, modifications to functional requirements, and changes to system design tend to compromise previously implemented security constraints and to hinder the ability to perform thorough security assurance reviews. Moreover, in Agile environments, access control requirements are typically embedded within evolving natural language artifacts, such as user stories, rather than being formally specified. This makes it difficult to systematically capture, verify, and maintain access control policies over time, and hinders the ability to perform thorough security assurance reviews throughout the development lifecycle.

Our contributions are as follows:

• We propose a novel framework for extracting access control policies from user stories in agile software development, leveraging RAG and LLMs. We take user stories as inputs, with the goal of deriving access control policies that are expressed in natural language within the requirements described by these stories. By analyzing these textual artifacts, our framework identifies the access control rules embedded in the product requirements.

• We design an automated pipeline that identifies candi-date access control rules from user stories. This allows teams, particularly SOC analysts or security reviewers, to evaluate whether these early requirements align with the actual implemented policies, identify potential gaps, and support more informed access reviews.

• We introduce a specialized retrieval mechanism to enhance the precision of access control rule extraction, minimizing irrelevant data and improving contextual understanding.

• We develop an intelligent agent to assist Security Operations Center (SOC) analysts in querying and validating access control rules efficiently.

• We provide a comprehensive evaluation of our frame-work using real-world agile software development datasets, demonstrating its effectiveness in policy extraction accuracy and system scalability.

• We provide a score representing the confidence of the model in the selected answer span, with higher values indicating greater confidence.

This article is organized as follows: In Section II, we provide background information on agile software devel-opment, RAG, and LLMs. In Section III, we present related work along with the challenges and limitations of existing contributions. In Section IV, we propose our framework. In Section V, we describe the dataset used, followed by the results, discussion, and analysis. In Section VI, we conclude the paper and outline future work perspectives.

II. BACKGROUND.

A. USER STORIES IN AGILE SOFTWARE DEVELOPMENT

Agile software development (ASD) has proven its effective-ness in delivering high-quality software by accommodating evolving requirements, reducing time to market, and aligning IT solutions with business needs. At the core of ASD are iterative practices such as planning, daily stand-up meetings, retrospectives, and iterative reviews, which facilitate contin-uous consideration of requirements and foster collaboration between product owners and development teams. These practices leverage user stories, a widely adopted artifact written in semi-structured natural language, to articulate software functionality from the user’s perspective.

User stories are concise and simple descriptions of a feature or functionality written from the perspective of an end user. They are commonly used in agile development to capture the requirements and goals of users in a way that is easy to understand and actionable for development teams. A typical user story follows the format:

‘‘As a [user type], I want to [goal or action], so that [reason or benefit].’’

This structure ensures clarity about who the user is, what they need, and why it matters.

Despite their advantages, several studies have shown that user stories are vulnerable to multiple interpretations and often fail to capture complete requirements. They are also prone to ambiguity,,, which can lead to imprecision and conflicting interpretations of the documented requirements. These issues frequently affect the access control rules governing the system under development.

B. LLM

Transformers are a type of neural network architecture that excel at handling sequential data: In a transformer, text is first converted to numerical representations called tokens, which break down sentences into smaller, manageable units. Each token is then converted into a vector using a word embedding table, which assigns a unique vector representation to each token based on its position in a multidimensional space that captures semantic relationships. This vectorized form enables the transformer to perform mathematical operations on the tokens, allowing it to understand not just individual words but also the context in which they appear. Through mechanisms like self-attention, transformers weigh the importance of each token relative to others in the sequence, helping the model capture contextual relationships and dependencies regardless of token distance.

This powerful combination of tokenization, embedding, and attention enables transformers to perform exceptionally well on tasks in natural language processing, such as translation, summarization, and text generation.

Expanding on the revolutionary advancements of trans-formers, large language models (LLMs) have emerged as highly advanced AI systems trained on extensive corpora of text. A key strength of LLMs lies in their ability to generalize to new tasks from only a few examples or textual instructions. By leveraging the transformer architec-ture, LLMs have pushed the boundaries of natural language processing, powering groundbreaking applications such as ChatGPT, which was released in 2022.

C. RAG

Despite their impressive capabilities, LLMs face notable lim-itations. They struggle with domain-specific or knowledge-intensive tasks and often generate ‘‘hallucinations’’, where responses include incorrect or fabricated information. This issue is particularly pronounced when queries require information beyond the training data or up-to-date knowl-edge, underscoring the need for complementary strategies to enhance their reliability and accuracy.

One such strategy is Retrieval-Augmented Generation (RAG), which addresses these challenges by integrating external knowledge into the generation process. RAG retrieves relevant information from external databases using semantic similarity techniques, combining it with the intrinsic knowledge of LLMs. This approach significantly improves the accuracy and credibility of generated responses, partic-ularly in knowledge-intensive or domain-specific contexts. Additionally, RAG facilitates continuous updates to the knowledge base, making it adaptable to dynamic environ-ments and specialized domains. By integrating retrieval with generation, RAG proves especially effective in domains requiring context-specific and explainable outputs, such as cybersecurity and access control, where accurate and context-aware decision-making is critical.

Advanced RAG enhances retrieval and contextual integration by refining indexing techniques, incorporating fine-grained segmen-tation, metadata, and alignment optimization. It further improves query handling through optimization methods like rewriting and expansion, ensuring user queries are clear and well-suited for retrieval tasks. Furthermore, post-retrieval processes focus on re-ranking retrieved chunks and compressing context to emphasize the most relevant information. Figure 1 depicts Advanced RAG pipeline: The process begins with an indexing phase, where documents are split into smaller chunks, tokenized, and converted into vector embeddings, which are stored in a vector database for efficient retrieval. During query processing, a pre-retrieval step applies techniques such as indexing, query rewriting, and query expansion to refine the input query.

The retrieval module then extracts relevant document chunks based on semantic similarity, followed by a post-retrieval reranking step that prioritizes the most useful information. Finally, the refined content is incorporated into a system prompt and processed by a Large Language Model (LLM) to generate a response. This pipeline improves information retrieval by ensuring that the LLM operates on contextually relevant and structured data.

D. MOTIVATION AND PROBLEM STATEMENT

In Agile development context, access control requirements are frequently embedded within user stories, rather than articulated through formal specifications. This lack of structure complicates the systematic extraction, validation, and maintenance of access control policies, thereby impeding comprehensive security assurance throughout the software development lifecycle.

While LLMs offer promising capabilities for understand-ing and processing natural language, they face notable challenges in security-critical applications. LLMs may pro-duce false or irrelevant outputs when context is insufficient, rely on outdated or overly generic training data, and reference informal sources. These limitations are particularly con-cerning in cybersecurity, where precision and reliability are paramount. RAG addresses these limitations by integrating retrieval mechanisms that source relevant, authoritative, and context-specific information from predetermined knowledge bases. In our proposed framework for extracting access control rules from user stories and other requirements’ documentation, RAG enhances the accuracy and relevance of LLM-generated outputs by grounding responses in authori-tative documents.

Furthermore, RAG provides transparency by disclosing the sources of the retrieved information, thereby offering insight into the reasoning process behind generated responses and increasing trust in the system’s recommendations.

III. RELATED WORK.

A. LITERATURE REVIEW OF ADVANCED AI IN ACCESS CONTROL

1) GNNs.

Traditional access control models face significant chal-lenges in big data environments due to the large volume, dynamic nature of resources, and complex subject-object relationships. These models face the problems of heavy policy configuration workload, limitations in adapting static policies to dynamically generated resources, and difficulties in handling complex security constraints among diverse user relationships and data types. In, the authors introduce a link prediction model based on dual-source learning of graph neural network (LPMDLG). Their approach enhances pre-dictive accuracy by incorporating both topological structure and node embedding features through a dual-source learning approach.

The LPMDLG framework learns topological features from directed enclosing subgraphs, accounting for edge direction and type, and uses neighbor subgraphs to improve node embeddings by considering relational factors. Compared with existing models, LPMDLG demonstrates improved accuracy in predicting edges in complex relation-ship graphs. Experimental results validate its effectiveness for dynamic and robust access control in big data systems.

In, the authors introduced an advanced key prove-nance identification framework based on heterogeneous graph neural networks (KPI-HGNN) to address dynamic attribute generation and multi-source aggregation challenges arising from big data resources in dynamic access control sce-narios. By integrating a community detection algorithm, their framework autonomously clusters data into distinct regions, addressing resource distribution imbalances. A key node identification method is then employed within each commu-nity, enhancing rule precision by minimizing redundant paths. Finally, an automated dependency path discovery algorithm enables dynamic rule generation, outperforming baseline models in accuracy, resource coverage, and efficiency. This solution effectively mitigates the challenges of redundant data and uneven key node identification.

In, the authors introduce a novel semi-supervised learning framework using heterogeneous graph neural networks (HGNNs) to enhance access control decision-making in modern information sys-tems. Their approach embeds complex relationships between users and resources by embedding both organizational and operational structures in node representations. By employing self-supervised link prediction on a heterogeneous access control graph, the authors learn enriched embeddings that improve access control performance in a supervised model. Their experiments on the Amazon access control dataset validate the framework’s effectiveness, surpassing tradi-tional models.

In, the authors introduce IAMPERE, a novel solution designed to address Privilege Escalation (PE) misconfigu-rations within cloud platforms. In their approach, MaxSAT is used to optimize the repair of PE vulnerabilities within IAM configurations by finding a minimal set of changes that can resolve these vulnerabilities. To enhance efficiency, the approach leverages a GNN that models IAM configurations as permission flow graphs, helping to prune the search space for the MaxSAT solver. This combination allows IAMPERE to produce an approximately minimal patch that effectively addresses misconfigurations with minimal computational load. As a result, it has the potential to efficiently enhance the integrity of access control policies within cloud environments, where secure and precise access control is essential to prevent unauthorized access and potential data breaches.

In, the authors introduce a novel approach to access control within Zero Trust Architecture (ZTA) by leveraging a hash-based, multidimensional GNN model. Their approach models user behavior patterns, combining feature hashing and multidimensional GNNs to achieve deep feature extraction and accurate access decisions.

2) LLMs AND RAG.

The authors in present a novel approach that leverages Large Language Models (LLMs) like ChatGPT and Google Bard for secure software construction. Their method uses for-mal specifications, specifically the Java Modeling Language (JML), as a foundation to guide LLMs in producing code with embedded security features. They created a modular pipeline, integrating automated verification tools to ensure generated code aligns with security specifications, using Role-Based Access Control (RBAC) as a testing model. By evaluating this pipeline across two case studies, including a banking application and a RBAC API, they demonstrated how LLMs can support developers in producing secure applications with valuable insights for future applications of LLMs in secure coding.

In, the authors introduce Intent-Based Access Control for Databases (IBAC-DB), a novel framework designed to enhance the management of access control policies in enterprise databases. By utilizing a natural language access control matrix (NLACM), the proposed system allows admin-istrators to express access control policies more precisely, facilitating automatic synthesis of database access control primitives. The implementation, called LLM4AC, demon-strated superior performance on benchmarks, achieving high accuracy and F1 scores compared to traditional methods. The research highlights the effectiveness of LLM4AC in automating policy implementation and auditing, while also addressing additional deployment requirements through the proposed systems RHieSys and DePLOI.

Overall, the findings suggest that IBAC-DB represents a significant advancement in automating and ensuring compliance in database access control, reducing the risks associated with misconfigurations.

In, the authors introduce ACFIX, an innovative tool designed to effectively repair AC vulnerabilities in smart contracts. Recognizing the limitations of existing detection tools in automatically addressing these vulnerabilities, they leverage the capabilities of LLMs, specifically GPT-4. ACFIX employs a two-phase approach, beginning with the mining of a comprehensive taxonomy of common RBAC practices from over 344,000 smart contracts, categorizing essential role-permission pairs. In the second phase, the tool utilizes a Multi-Agent Debate mechanism to guide GPT-4 in accurately identifying the appropriate role-permission pairs for vulnerable code and generating effective patches. The authors evaluate ACFIX’s peformance using a benchmark dataset, achieving an impressive repair rate of 94.92%.

In, the authors introduce RAGent, a retrieval-based access control policy generation framework based on lan-guage models. Their approach identifies access requirements from high-level requirement specifications.

B. ANALYSIS AND DISCUSSION

This section synthesizes the current advancements in advanced AI-driven approaches for contextualized access control, with a focus on the potential of GNNs and LLMs to enhance security in complex, dynamic environments. By analyzing recent studies, we examine the performance of these approaches in addressing limitations of traditional access control systems and their ability to adapt to evolving security requirements.

1) GENERATION OF DYNAMIC ACCESS CONTROL POLICIES.

Traditional static access control policies often fall short when addressing the needs of highly dynamic and context-sensitive environments, such as those found in cloud computing, IoT ecosystems, and multi-user collaborative platforms. These environments often involve fluctuating access requirements, diverse user roles, and rapidly changing data sensitivity levels, all of which challenge the rigidity of static policies.

Dynamic access control policies, by contrast, can adapt in real-time to context changes. This adaptability mini-mizes the risk of unauthorized access while improving user experience by granting access based on up-to-the-minute context, rather than pre-set, static rules. Additionally, the generation of dynamic policies enhances resilience against insider threats and cyberattacks, as access decisions can quickly adjust in response to anomalous behavior or emerging threats.

Recent research has made significant strides GNNs and LLMs to enable dynamic access control policies in complex and evolving environments. For example, the LPMDLG model introduced by enhances prediction accuracy in access control by learning both structural and relational features in graph-based data, making it highly suitable for big data systems with complex relationships, presents the KPI-HGNN framework, which addresses dynamic attribute generation and data clustering challenges, improving the precision of access rules by reducing redundant paths. Similarly, the authors of utilize multidimensional GNNs to model user behavior and enable accurate access decisions.

By using heterogeneous graph neural networks (HGNNs) to embed complex relationships between users and resources, the approach used, improves policy generation through self-supervised learning, creating enriched embeddings that lead to more accurate and dynamic access control decisions. Complementing these GNN-based models, introduce IBAC-DB, an LLM-powered intent-based access control framework that uses natural language for policy generation and management in enterprise databases. Together, these works showcase the potential of LLMs and GNNs to automate and refine dynamic access control in complex, data-rich environments.

2) ENHANCEMENT OF THE INTEGRITY AND THE SECURITY.

OF ACCESS CONTROL, AC VULNERABILITIES

Enhancing access control integrity and security is essential for preventing unauthorized access and mitigating vulnera-bilities like misconfigurations and insider threats. Traditional access control systems often lack the flexibility to adapt to complex, dynamic environments. Leveraging LLMs and GNNs offers a solution by enabling context-aware, adaptive policy generation and real-time decision-making. Recent advancements utilizing GNNs and LLMs have signifi-cantly contributed to enhancing the integrity and security of access control systems, addressing vulnerabilities like misconfigurations and privilege escalation. For instance, the IAMPERE solution leverages GNNs to model identity and access management configurations as permission flow graphs, optimizing privilege escalation vulnerability repairs with minimal computational load.

This approach improves the accuracy of access control policies within cloud environ-ments, reducing the risk of unauthorized access. Additionally, the integration of LLMs in secure software construction, as demonstrated in the use of formal specifications for Role-Based Access Control (RBAC), helps developers embed security features into their applications, enhancing the reliability of access control implementations. Finally, the ACFIX tool employs LLMs like GPT−4 to automate the repair of AC vulnerabilities in smart contracts, achieving high accuracy in role-permission patching, further demonstrating how LLMs and GNNs can address AC vulnerabilities effectively.

3) CHALLENGES AND LIMITATIONS.

While the use of GNNs and LLMs in access control offers significant advancements, several challenges remain. One key issue is data privacy, as both GNNs and LLMs often require access to sensitive data to learn and make accurate predictions, raising concerns about unauthorized exposure or misuse of personal or confidential information. Another challenge is the complexity of system integra-tion, as implementing these models within existing access control systems requires seamless integration with diverse technologies, platforms, and data sources, which can be resource-intensive and prone to technical hurdles. Ethical considerations also play a critical role, as these models may unintentionally perpetuate biases present in training data, leading to unfair or discriminatory access control decisions.

Additionally, the scalability of GNNs and LLMs in large, dynamic environments poses another challenge, as these models require significant computational resources and might struggle to maintain efficiency as the scale of the system or dataset increases. Furthermore, model interpretability remains an ongoing issue, as understanding the decision-making process of these advanced models is often difficult, which could hinder trust and accountability in security-critical applications. Addressing these challenges is essential to ensure the effective and ethical deployment of GNNs and LLMs in access control systems.

To address these challenges, we propose a novel frame-work that leverages LLMs and RAG for extracting access control rules from structured and unstructured data sources. By integrating retrieval mechanisms, our approach enhances interpretability by grounding LLM outputs in explicitly retrieved context, reducing reliance on opaque model-generated responses. Additionally, this retrieval-based strat-egy minimizes direct exposure to sensitive data, mitigating privacy concerns by focusing on relevant policy fragments.

While recent works have applied LLMs in the context of secure code generation, access control policy synthe-sis, and smart contract repair, our proposed framework introduces a new application domain: the extraction of access control rules from user stories in agile software development.

IV. PROPOSED FRAMEWORK.

We design a framework to extract and process access control policies from user stories and software requirements specification. We begin by sourcing input from user stories and specifications, which serve as knowledge sources. These inputs are segmented into manageable chunks or tokens through a splitting and tokenization process. Each chunk is then encoded into vector embeddings, which capture their semantic meaning. These embeddings are stored in a vector database, enabling efficient retrieval of relevant chunks based on context. The retrieved chunks are sub-sequently used to construct a system prompt, facilitating advanced ML-driven analysis or reasoning for specific access control tasks. Because access control requirements evolve alongside the system, the framework supports continuous updates by allowing reinjection of modified or newly created user stories.

This enables the extraction pipeline to remain aligned with the most current version of the system’s requirements. By integrating RAG techniques, we aim to ensure a contextually accurate and scalable access control policy extraction. Figure 2 presents an illustrative workflow using RAG and LLMs for context-aware access control question answering.

We split the process into these phases:

1) Creating Index: We use an indexing pipeline to fetch.

data, process it, and load it into the document store.

2) Embedding Generation: This step generates embeddings.

from the input data according to the model used.

3) Building an Extractive QA Pipeline: this process.

combine all modules (Retriever-Ranker-Reader), it turns a query into a vector and returns answers to that query, as well as their location in the source document, and a confidence score.

4) Generative QA using LLMs: Unlike the extractive.

QA pipeline, which identifies and extracts exact text spans from source documents, this phase generates detailed responses by synthesizing information from the retrieved and ranked data.

Algorithm 1 outlines the steps involved in our approach.

Load a pre-trained Sentence-Transformer model For each user story ui ∈ U, generate the embedding e(ui) ← Model(ui)

For each predefined access policy pj ∈ P, generate the embedding e(pj) ← Model(pj)

Step 3: Building an Extractive QA Pipeline

Combine Retriever, Ranker, and Reader components into the extractive QA pipeline

For each query (user story or extracted entity), use the retriever to fetch relevant passages from the document store Rank the fetched passages based on relevance

Use the reader to extract exact text spans from the ranked passages

Step 4: Generative QA using LLMs

Use a LLM to provide context-aware answers by integrating multiple passages

Step 5: Extract Access Control Policy Details

From the generated response, extract roles, actions, and conditions specified in the access control policies Output: List of extracted access control policies R = {r1, r2..., rk }

V. RESULT AND ANALYSIS.

A. DATASETS

We used a collection of 22 datasets, this collection includes 21 web applications, each containing between 50 and 130 user stories, amounting to over 1,600 user stories in total. These applications span diverse domains such as financial management, healthcare, administrative management, and others. The datasets were obtained online or provided by software companies with permission for disclosure,1 Table 1 provides an overview of the datasets collection we used in our work.

B. EXPERIMENTAL SETTINGS

The experiments were conducted using the Haystack framework for natural language processing and question-answering tasks. To efficiently process the data and accelerate computations, the experiments were performed on Google Colab with GPU support enabled. This setup provided the necessary computational power for RAG, and the execution of LLMs. The combination of Haystack’s robust capabilities and the accessibility of Google Colab facilitated the seamless development and evaluation of the framework.

C. RESULTS

We illustrate obtained result using the Federal Spending Transparency project, which pertain to the website that is used to share publicly the spending data for the U.S. government. The website was created because of the Digital Account-ability and Transparency Act of 2014 (DATA Act). Current and recent snapshots of federal spending related websites, including many more projects than the one described in the shared dataset, can be found here.2 As an input, we use the file g02-federalspending.txt which contains user stories regarding this project, figure 3.

To illustrate the nature of the user stories used in our experiments, we include the following example:

‘‘As an agency user, I want to be able to include a large number of flexfields without performance impact.’’

This user story implies a requirement that can be mapped to an access control rule, where an agency user needs write or submit access to submission forms with many flexfields. The framework processes such user stories to extract the subject (agency user), the action (include), and the resource (flexfields), forming a role-resource-action mapping that can guide SOC analysts in evaluating access requests.

We use the model ‘‘all-distilroberta-v1’’ to generate embeddings.

We experiment with representations produced by all-distilroberta-v1 (DistilRoberta) model, a distilled instance of roberta-base fine-tuned using the training archi-tecture from Reimers and Gurevych on a 1B sentence-pair corpus. Its ability to convert sentences and paragraphs into dense vectors makes it a powerfull tool for tasks like clustering, semantic search, furthermore its contrastive learning objective allows it to capture the nuances of language, making it a great choice for critical information retrieval such our use case regarding access policies.

Scoring: The used score reflects the model’s confidence in the extracted answer span. It is computed using the probabilities assigned to the start token and end token of the predicted answer span.

Score = Pstart(i) · Pend(j) where:

• Pstart(i) is the probability of the token at position i being the start of the answer span.

• Pend(j) is the probability of the token at position j being the end of the answer span.

• i, j are the indices in the document corresponding to the predicted start and end tokens, respectively.

The probabilities Pstart(i) and Pend(j) are computed by the model using a softmax function over all tokens in the document, as follows: exp(ej) exp(si), Pstart(i) = Pend(j) = PN PN k =1 exp(sk) k =1 exp(ek) where:

• si and ej are the raw scores (logits) for the start and end positions output by the model.

• N is the total number of tokens in the document.

• exp(x) represents the exponential function applied to x.

The final score represents the model’s confidence in the selected answer span, with higher values indicating greater confidence.

Table 2 depicts the obtained score when extracting access privileges per dataset for a specific role. The framework provides answers regarding the specified profile and the actions she/he can perform according to the user stories. The results demonstrate consistently strong performance, with accuracy scores ranging from 67.38% to 84.21%. Notably, the model achieved the highest accuracy for ArchivesSpace dataset (84.21%) and CUL dataset (82.58%), indicating its ability to capture well-structured access poli-cies. Most roles scored above 75%, showcasing the model’s reliability across different contexts.

D. DISCUSSION AND ANALYSIS

The proposed framework integrates three powerful compo-nents to achieve its objectives. The InMemoryBM25Retriever efficiently retrieves relevant text snippets by leveraging lexical term-based matching. The sentence-transformers/all-distilroberta-v1 model encodes the semantic meaning of the retrieved text, enabling a deeper contextual understanding. Finally, the Transformers Similarity Ranker refines the results, ensuring that only the most contextually relevant mappings are presented to the user.

By leveraging the strengths of these tools, the framework extracts for a given profile her/his access privileges according to the knowledge sources. It also provides a relevance score which represents how well the answer fits the query context after re-ranking. Achieved Max scores for each dataset are between 67.38% and 84.21%. While the initial results are encouraging, it is important to acknowledge the limitations of the proposed framework. The current implementation depends heavily on the quality and consistency of user stories. Additionally, the framework does not yet include a robust validation mechanism before applying generated policies, which is critical given the sensitivity of access control in real-world systems. Introducing automated or semi-automated validation steps could significantly enhance the reliability and safety of deployment.

Beyond these limitations, the framework could be further enhanced by exploring alternative retrievers, rankers, and language models that may offer improved performance in access control contexts. Incorporating pre-processing techniques tailored to access control vocabulary could also help refine the extraction process and reduce ambiguity. Furthermore, while our current approach infers policies based on user roles, future work could focus on fine-grained access control rule extraction based on user and resource attributes to support attribute-based access control (ABAC). Evaluating the framework within real-world SOC environments would provide valuable insights into its practical applicability and guide further refinements.

VI. CONCLUSION.

In this article, we discussed the impact of changes in soft-ware requirements specification on access control policies. We proposed a method for extracting access control rules from user stories in agile software development environ-ments. For a given profile, the proposed approach provides access privilege along with a relevance score using a QA pipline. The goal of our framework is to support SOC analysts and other stakeholders in managing and enforcing access controls more effectively. In an era where cybersecurity threats are increasingly sophisticated, the ability to derive actionable insights from textual documentation is crucial for enhancing system security and operational efficiency.

While challenges remain, the framework represents a significant step toward integrating cutting-edge AI tools into cyberse-curity workflows, empowering SOC analysts to meet the demands of modern security environments.