Scientists are increasingly running their research on tools they cannot fully see inside, according to a new study published August 15. AI models, satellite data products, wildlife trackers and commercial survey services now shape findings in ecology and conservation, but researchers often lack access to the algorithms, training data and raw inputs behind the outputs - a transparency gap the authors say threatens reproducibility and trust in science.
The international team of researchers calls these systems "black boxes." They process enormous amounts of information, monitor biodiversity across continents, and reveal patterns that once sat out of reach. Yet the methods that produce those results stay largely hidden, often under commercial lock.
"Many of these tools represent true black boxes, by keeping the processes behind those results largely hidden," said Ivan JariΔ, lead author of the study and researcher at the University of Paris-Saclay. "They are often owned by private companies that intentionally limit access to information about how their systems operate or process data, guided by proprietary constraints and commercial aims."
The problem goes beyond AI
Large language models and other AI systems are the most visible example. Scientists use them to analyze large datasets, interpret satellite imagery and model ecosystems. But researchers often get little or no access to training data, algorithms or direct tests of the system, and cannot tell why a model produces a particular output. The authors warn that as AI grows more capable and autonomous, findings will get harder to interpret and verify.
The issue reaches older technologies too. Many remote sensing products rely on proprietary processing researchers cannot fully access. Some wildlife tracking devices deliver processed animal locations but withhold raw underlying data. Online platforms such as search engines and social media, now common sources for studying biodiversity and human-nature interaction, run on hidden algorithms and changing policies that can quietly inject bias into those datasets.
Even social surveys are affected. Scientists increasingly hire private companies to recruit participants and handle surveys, with limited information on how respondents are chosen, how quality is maintained, or whether AI bots have tainted responses.
"This problem is not simply due to commercial and proprietary issues," said Professor Karen Anderson from the University of Exeter, another author. "Modern scientific tools are also becoming so complex that users, and in some cases even their developers, may struggle to fully complete and understand how they operate."
Publication pressure makes it worse
The dependence on opaque tools grows alongside pressure on scientists to produce more research, work faster and cope with expanding data sets and pressing environmental crises. The researchers warn of monopoly risk, weakening open-science efforts and susceptibility to manipulation. Most damaging of all, they say, is the hit to reproducibility: if key analytical steps cannot be inspected or repeated, confidence in findings fades.
The authors propose concrete fixes: choose open-source software and hardware when possible, benchmark proprietary tools against transparent datasets, compare results across multiple methods, and document training data, pipelines, versions, settings and limitations.
Human oversight stays central
"Human oversight should remain central throughout the research process, especially since it is the study authors who must take responsibility for any errors and uncertainties produced by the use of black-box tools in their work," said Michael Bertram from the Swedish University of Agricultural Sciences and Stockholm University.
He also called for stronger open-science regulations that give researchers better access to digital platforms and their underlying data. But some black boxes, the authors note, will stay resistant. Scientists should stay alert to the trade-offs and the risks of adopting such systems without questioning.
Why this matters for research professionals
Scientists contributing to any field that relies on AI, remote sensing, wildlife monitors or commercial survey data need to make a few practical moves now. Document the exact versions and settings you used - write it down and make it impossible to miss it in your methods section. When you can, choose an open tool with a comparable, replaceable alternative and help improve it instead of guarded by the ecosystem around proprietary products. Challenge AI vendors in procurement meetings and on survey front-end platforms about their training data and processing pipelines, because if they cannot answer at a scientific standard, that limitation belongs in your paper. These are not administrative burdens you can postpone - they are the difference between findings you can defend and results that cannot produce confidence in anyone outside your team.
Your membership also unlocks:
