Finding the Signal in the Noise: What building an AI assistant for nature credit design taught us about the limits of AI, and where the real work lies

©Diego Pérez - SPDA/Conservación Internacional

Practitioners developing nature credit projects face decisions that are both highly technical and hard to get right: Which framework applies to their context? What biodiversity outcomes should they monitor? How do they balance scientific rigor with what's actually deployable in the field? With more than 60 frameworks and standards in play, each with different requirements, crediting logic, and applicability, and no comprehensive resource to navigate them, getting these decisions wrong early is costly.

Conservation International, Earth Genome, and the Nature Tech Collective set out to test whether AI could help. After interviewing experts and practitioners across the biodiversity and finance landscape, mapping out the myriad questions that different people wrestle with at every stage of project development and across their different vantage points, and curating a small arsenal of key knowledge sources: credit frameworks, biodiversity monitoring protocols, and nature technology specifications, we produced a prototype AI Assistant called Eco.

Eco was a modular platform with specialist tools aligned to each stage of the user journey, each with specific instructions and pointing to a specific knowledge library.  We iteratively tested and refined as we went: Eco was originally too overbearing in its attempts to be helpful, so we moved from prescriptive prompting to more open-ended interactions. We found that standard RAG got too many things wrong too often, essentially getting lost in the dense, complex documentation- so the Earth Genome team developed a really clever pre-processing technique that converted input documents to JSON “rulesets.” This new JSON RAG or “JRAG” approach encoded the key characteristics and criteria of the input documents. The result was a more auditable trail for experts to interrogate how Eco arrived at different conclusions. 

The JRAG approach improved the fidelity with which the tool evaluated example projects against specific criteria, e.g., is this project eligible for a given framework?, but it still struggled to help users weigh frameworks against each other or explore tradeoffs between monitoring rigor and cost. The system tended toward vagueness or tried to fit every framework to the use case rather than helping users reason through the decision. JRAG solved an important problem that is relevant to many different assessment contexts, just not the problem our teams were really struggling with. 

There could be many reasons we didn’t get to the solution we were reaching for. Despite a fantastic team of engineers and experts across Conservation International, Earth Genome, and the Nature Tech Collective, this was a tiny pilot project with a minimal budget and tight timeline.  Our curated resource library was illustrative, not comprehensive.  And perhaps most importantly, there is no consensus on “what a good answer looks like”, so it’s incredibly difficult to evaluate success, much less improve it. There is no definitive decision logic nor do these questions have clear, objective, “right” answers. 

But we very much view this project as a success.  We set out to explore the question: “Can AI help practitioners navigate nature credit project design?” And we got an answer: AI holds significant potential, but we need to curate the foundational knowledge and define what good looks like. This work can't be done in isolation, it requires common benchmarks, open documentation of agent architectures, and collaboratively maintained knowledge bases across the sector. 

We realize that what’s possible changes by the day, and in the six months since we did this work, the field has already advanced. But these underlying needs- for foundational knowledge and clarity on what “good” looks like, persist.  

We are continuing to test and explore the potential of AI to support biodiversity monitoring and design decisions, and we’d love more partners in this adventure. If this is something that intersects with your interests, we’d love to connect!  

You can read the official 4-page summary here or dig into the full technical report produced by Earth Genome here. If you want to talk or have any questions, please reach out at aswanson@conservation.org.