The Special Inspector General for Afghanistan Reconstruction just released the 25th quarterly report (pdf) on US reconstruction efforts in Afghanistan. We are of course at a major crossroads in US involvement in Afghanistan, as US and NATO combat involvement are being phased out and Afghanistan assumes responsibility for its own security. Some US and international troops will remain in Afghanistan after the end of this year under the new Bilateral Security Agreement, but with Afghanistan in charge it is of utmost importance that the Afghan National Security Forces (ANSF) are fully staffed and functional so that they can take on their responsibilities. One of SIGAR’s key roles in its oversight activity through the years has been to collect and review information coming directly from ISAF, the International Security Assistance Force, concerning the recruitment, training and subsequent capabilities of ANSF. ISAF ostensibly is a NATO team but is of course dominated, both in command and in personnel, by the US military.

Suddenly, in the final SIGAR report before the current ISAF mission ends and operations move to the new arrangement, ISAF, and more specifically ISAF Joint Command, has decided to classify the reports it prepares on ANSF troop capability. Here is Inspector General John Sopko in his cover letter accompanying the quarterly report:

This quarterly report also examines the reconstruction effort across the security, governance, and economic sectors. In the security sector, SIGAR was deeply troubled by the decision of the International Security Assistance Force (ISAF) to classify the executive summary of the report that assesses the capability of the ANSF. For years, SIGAR has used the ISAF report as a primary metric to show Congress and the public the effectiveness of the $61.5 billion U.S. investment to build, train, equip, and sustain those forces. Prior to this quarter, aggregate data on the operational effectiveness of the ANSF were unclassified in the Regional ANSF Status Report (RASR) as well as its predecessors, the Commanders’ Unit Assessment Tool (CUAT) and the Capability Milestone rating system.

ISAF’s classification of the report summary deprives the American people of an essential tool to measure the success or failure of the single most costly feature of the Afghanistan reconstruction effort. SIGAR and Congress can of course request classified briefings on this information, but its inexplicable classification now and its disappearance from public view does a disservice to the interest of informed national discussion. Moreover, while SIGAR understands that detailed, unit-level assessments could provide insurgents with potentially useful intelligence, there is no indication that the public release of aggregated data on ANSF capabilities has or could deliver any tactical benefit to Afghan insurgents.

It is very difficult to see this move by ISAF as anything more than a blatant attempt to cover up massive failure on the part of the efforts to train Afghan troops to take over their own security functions. This move by ISAF follows previous efforts that also come off as attempts to game the system so that evaluation of the always-claimed “progress” is difficult to impossible. Note in Sopko’s letter that he refers to three different systems by which troop readiness has been analyzed and reported. First, we had the Capability Milestone system, which was replaced by the Commanders’ Unit Assessment Tool (CUAT) and the now-classified Regional ANSF Status Report has replaced CUAT.

In March of 2013, I pointed out SIGAR’s frustration with how ISAF was gaming the CUAT:

A related area in which SIGAR has found a disgusting level of dishonesty is in how the US goes about evaluating Afghan forces in terms of readiness. Because it became clear to the trainers in 2010 that they had no hope of achieving the trained and independent force size numbers that NATO planners wanted (and because SIGAR found that the tool they were using at the time was useless), they decided that the only way to demonstrate sufficient progress was to redefine the criteria for evaluating progress. From the report:

In 2010, SIGAR audited the previous assessment tool—the Capability Milestone (CM) rating system which had been in use since 2005—and found that it did not provide reliable or consistent assessments of ANSF capabilities. During the course of that audit, DoD and NATO began using a new system, the CUAT [Commander’s Unit Assessment Tool], to rate the ANSF. In May 2010, the ISAF Joint Command (IJC) issued an order to implement the new system which would “provide users the specific rating criteria for each [ANSF] element to be reported by the CUAT including leader/commander considerations, operations conducted, intelligence gathering capability, logistics and sustainment, equipping, partnering, personnel readiness, maintenance, communications, unit training and individual education, as well as the partner unit or advisor team’s overall assessment.”

Since the implementation of the CUAT, the titles of the various rating levels have changed, as shown in Table 3.3. In July 2012, the Government Accountability Office (GAO) raised concerns that the change of the title of the highest rating level from “independent” to “independent with advisors” was, in part, responsible for an increase in the number of ANSF units rated at the highest level. GAO also noted that “the change lowered the standard for unit personnel and equipment levels from ‘not less than 85’ to ‘not less than 75’ percent of authorized levels.” In a response to SIGAR last quarter, the IJC disagreed with GAO’s assessment, saying a change in title does not “equal a change in definition.” Since last quarter, the IJC has initiated a CUAT Refinement Working Group to standardize inputs and outputs in the areas covered by the assessments.

But it turns out that the CUAT itself was developed only when SIGAR initiated an audit (pdf) of the Capability Milestone rating system. So, twice, when SIGAR decided to audit the system for evaluating Afghan troop readiness, ISAF responded by developing a totally new system, creating a strong discontinuity in the ability to track Afghan troop readiness over time. And now that we are at the most important moment for Afghan troops to be ready, ISAF decides that any information at all on their readiness is classified, even though they have provided the very same information without classification for years.

When we drill down to the details about the classification that SIGAR provides in the report, we see in footnote 196 (page 94) that they were informed of the classification in response to a data call submitted to IJC on October 3 of this year. Noting this and the arguments that SIGAR provides that aggregate data on Afghan troop readiness should not provide any sort of strategic advantage to insurgents, I submitted the following question to SIGAR:

Over the years, ISAF Joint Command has changed both the tools and the criteria for reporting ANSF capabilities a number of times. As an outsider who has followed these changes carefully, I find it hard to ascribe these moves to anything other than efforts to obfuscate lack of progress on readiness by making reports over time hard to compare to one another.

Choosing to classify the report at this time would seem the ultimate extension of such practices and seems tantamount to an admission that ANSF are not ready for the challenges provided by the transition of forces that occurs under the new Bilateral Security Agreement. In the report, you provide a convincing argument that release of aggregate data on readiness would not provide a strategic advantage to insurgents. Were you given a justification for the classification by IJC? Was it described as being for the security of ANSF and preventing the publication of information that could be used strategically?

If you cannot comment on any reasoning given by IJC for the classification decision or were given no explanation, can you at least comment on whether the decision has any impact on your view of the credibility of IJC as an honest broker of information on ANSF capability?

Here is the answer I received from Inspector General Sopko:

As noted in SIGAR’s latest quarterly report, SIGAR is troubled by ISAF’s decision to classify the executive summary of the RASR. According to the RASR, “The [then-unclassified] EXSUM presents a synthesized analysis of observations and identified shortfalls, highlighting main findings and most pressing issues that hamper ANSF long-term sustainability.” When asked why the executive summary is now classified, SIGAR received the following answer via CENTCOM:

“Last month, one of the IJC theater disclosure officers reviewed the ‘Executive Summary’ portion of the RASR and determined the classification was higher than the original ‘NATO/ISAF UNCLASSIFIED’. Specifically, the portions that discussed any of the ANSF capabilities (such as issues) was classified at least Restricted. The tables that listed the count of units by rating (Fully Capable, Capable, etc.) was classified ‘SECRET’.”

ISAF’s response suggests that the decision to classify the executive summary was based on concerns that it could be used strategically against the ANSF, although that was not explicit in ISAF’s response. SIGAR maintains that the executive summary, or a similar product that tracks aggregated data, should be provided so that SIGAR can continue to report on the ANSF’s progress as a primary indicator of how effective the United States’ considerable investment has been. A product of this type would not need to identify specific battalions or units, but only give a high-level overview of readiness.

We cannot speculate on why IJC classified the executive summary. We are continuing to work with ISAF to find a solution to this problem that balances both ISAF’s concern for security with SIGAR’s concerns for transparency and accountability to U.S. taxpayers.

So while Sopko diplomatically would not address whether IJC’s decision to classify the data affected his view of ISAF’s honesty in providing information, it would seem significant that Sopko emphasized that SIGAR is “troubled” by the decision. And again, since it seems to be an extension of the practice of ISAF to change the evaluation tool each time it starts to receive extra attention, now that we are at the most important moment for ANSF, all information on their readiness suddenly being withdrawn appears to me to be highly suspect and disingenuous.

It also seemed significant to me that the Commander of ISAF, General John Campbell, took over from Joseph Dunford on August 26 of this year. Further, the head of IJC, Lieutenant General Joseph Anderson, assumed that command in February.

That prompted me to submit this question:

It appears from the footnotes in the report that you may have first learned of the classification decision in response to data calls dated September 30 and October 3. I note that General Campbell assumed command of ISAF August 26 and IJC Commander Lt. General Anderson has only been in that post since February. Do you have any reason to believe that the classification decision was driven by either or both of these personnel changes?

[Note: at the time I submitted the question, I misread footnote 197 to be from this year, so the question should only refer to October 3.]

Here is the answer from Sopko:

SIGAR learned about the change of classification of the executive summary of the RASR through our quarterly data call. While all information was provided to SIGAR, it was not provided in a way that SIGAR could use in a public report. SIGAR raised its concern with CENTCOM immediately and received the response that the decision was made by an unnamed “IJC theater disclosure officer.” At this point, SIGAR cannot speculate if the commanders of ISAF or IJC had any role in the change of classification, or if they are yet aware of this change. SIGAR is working with ISAF to find a solution to this problem that balances both ISAF’s concern for security with SIGAR’s concerns for transparency and accountability to U.S. taxpayers.

Let us hope that Sopko and SIGAR are successful in getting the “transparency and accountability” they seek. Sadly, though, we are just as likely to learn sad truths about ANSF readiness when we see how they respond to threats when acting on their own. Let’s hope they have been trained to a higher state of readiness than the large numbers of Iraqi troops who fled rather than confront ISIS, but it is difficult to expect anything better than that.