Page 2 of 3

Re: We NSA Now

PostPosted: Fri Jan 11, 2013 7:14 pm
by General Patton ... al-muscle/
As Occupy Wall Street swelled last year, many commentators asked: what’s the point? While this question was occasionally cynical, it did highlight that the crackling energy of protest lacked direction. Now, more than a year after OWS became a truly global movement, there may be less overwhelming force but more incisiveness.

Two recent campaigns exemplify Occupy’s growing power as a local organizer and media force: the rebuilding efforts after Hurricane Sandy and protest efforts against the Keystone XL Pipeline. We’ll look at both operations through the lens of Recorded Future to see how events developed and the role that Occupy has played.

The fact that a protest movement known for ragtag assemblies in public parks has been steadily organizing and contributing to a humanitarian relief campaign for nearly a month now speaks to its stamina and operational capability.
Occupy has evolved from a philosophical, loosely knit protest movement into something much more powerful. In just the last two months, we’ve seen effective mobilization at the community level (Operation Sandy) and amplification of causes of choice (Tar Sands Blockade, Black Friday) through the massive global audience Occupy has engaged.

One can still ask, what is the point of Occupy? Do Occupy groups merely adopt prior causes of community and environmental groups, or do they lead the way? It doesn’t matter; the Occupy Movement “voice” has become a global megaphone that transcends the divergent local ability to conduct operations. As reported by Fortune: Samantha Corbin, an Occupy Sandy site coordinator said, ‘The relationships we built through Occupy Wall Street are a huge reason why we’ve been able to scale so fast.’ The Occupy Movement has found its legs and cannot be ignored.

This next entry is particularly important when combined with extended social network mapping. Individual tweets are absolutely worthless, only taken in aggregate do they become useful. ... e-in-gaza/
Following up our recent post highlighting the escalation of conflict between Israel and Gaza we want to look at the earliest signs that the attack – both physical and cyber – was growing. We’ll tell the story in a series of images showing social media and news content analyzed in Recorded Future.

The first serious signals suggesting Gaza expected a more significant physical conflict showed up on November 11 when Hamas’ interior ministry called for an evacuation of all offices.

The first indications of the evacuation were revealed on Twitter and Facebook.

The earliest reports came from Dima-Gaza on Twitter (the earliest of which was removed) along with other mentions from Richard Dufek, who has been keeping a running report of the latest on Twitter, and his related Facebook group International communities against Israel:



The first news channel to break the story came after reports on Twitter via YNet although there was only light uptake on the news elsewhere in the mainstream media.

Speculation: Data overload does not exist. The problem is most people rely too much on their working memory when processing data, which limits to 5-9 pieces at a time as I discussed earlier. Visual aids are known to be very useful in working with this process. 3d displays and interfaces will likely drastically reduce interface friction. (see ... -on-chess/ ) ... athematics
The high point of the abacus calendar is the All Japan Soroban Championship, which took place earlier this year in Kyoto.

And the high point of the championship is the category called “Flash Anzan” – which does not require an abacus at all.

Or rather, it requires contestants to use the mental image of an abacus. Since when you get very good at the abacus it is possible to calculate simply by imagining one.

In Flash Anzan, 15 numbers are flashed consecutively on a giant screen. Each number is between 100 and 999. The challenge is to add them up.

Simple, right? Except the numbers are flashed so fast you can barely read them.

I was at this year’s championship to see Takeo Sasano, a school clerk in his 30s, break his own world record: he got the correct answer when the numbers were flashed in 1.70 seconds. In the clip below, taken shortly before, the 15 numbers flash in 1.85 seconds. The speed is so fast I doubt you can even read one of the numbers.

In the present study, we examined cortical activation as a function of two different calculation strategies for mentally solving multidigit multiplication problems. The school strategy, equivalent to long multiplication, involves working from right to left. The expert strategy, used by “lightning” mental calculators (Staszewski, 1988), proceeds from left to right. The two strategies require essentially the same calculations, but have different working memory demands (the school strategy incurs greater demands). The school strategy produced significantly greater early activity in areas involved in attentional aspects of number processing (posterior superior parietal lobule, PSPL) and mental representation (posterior parietal cortex, PPC), but not in a numerical magnitude area (horizontal intraparietal sulcus, HIPS) or a semantic memory retrieval area (lateral inferior prefrontal cortex, LIPFC). An ACT-R model of the task successfully predicted BOLD responses in PPC and LIPFC, as well as in PSPL and HIPS. w

Calculating prodigies are individuals who are exceptional at quickly and accurately solving complex mental calculations. With positron emission tomography (PET), we investigated the neural bases of the cognitive abilities of an expert calculator and a group of non-experts, contrasting complex mental calculation to memory retrieval of arithmetic facts. We demonstrated that calculation expertise was not due to increased activity of processes that exist in non-experts; rather, the expert and the non-experts used different brain areas for calculation. We found that the expert could switch between short-term effort-requiring storage strategies and highly efficient episodic memory encoding and retrieval, a process that was sustained by right prefrontal and medial temporal areas.

SCADA Issues: ... ces-010913
Never underestimate what you can do with a healthy list of advanced operator search terms and a beer budget. That's mostly what comprises the arsenal of two critical infrastructure protection specialists who have spent close to nine months trying to paint a picture of the number of Internet-facing devices linked to critical infrastructure in the United States. It's not a pretty picture. The duo ... have with some help from the Department of Homeland Security (PDF) pared down an initial list of 500,000 devices to 7,200, many of which contain online login interfaces with little more than a default password standing between an attacker and potential havoc. DHS has done outreach to the affected asset owners, yet these tides turn slowly and progress has been slow in remedying many of those weaknesses. ...The pair found not only devices used for critical infrastructure such as energy, water and other utilities, but also SCADA devices for HVAC systems, building automation control systems, large mining trucks, traffic control systems, red-light cameras and even crematoriums."

Those results came largely from a series of automated scripts run against the Shodan search engine. Shodan was created for the purpose of finding servers, routers, network devices and more that sit online. Users can filter searches to find specific equipment by manufacturer, function and even where they’re located geographically. A 2010 advisory on Shodan pointed out that the availability of the search engine greatly reduces the resources attackers require to find these privately owned assets. ... cking.html
Undertaken by the Dutch research lab TNO Defence, based in The Hague, the water industry study examined the security measures taken by the 10 companies that control the Netherlands’ drinking water. At issue are the Supervisory Control and Data Acquisition Systems (SCADAs) which, at a water plant, control processes like water intake, purification, quality control and pumping to homes.

A SCADA sends instructions to shopfloor machines like pumps, valves, robot arms and motors. But such systems have moved from communicating over closed networks to a far cheaper conduit: the public internet. This can give hackers a way in. Eric Luiijf of TNO Defence and his colleagues found a litany of insecure “architectural errors” in the waterworks’ SCADA networks (International Journal of Critical Infrastructure Protection, DOI: 10.1016/j.ijcip.2011.08.002).

Some firms did not separate their office and SCADA networks, allowing office hardware failures, virus infections and even high data traffic to potentially “bring down all SCADA operations”. While remote internet access to SCADAs is supposed to be possible only with strict security controls, the researchers found this was often not the case. And some water firms allowed third party contract engineers to connect laptops to their SCADA network with no proof they were running up-to-date antivirus software. Indeed, it has emerged that a US contractor logging on to check the Illinois water plant from Russia, while he was away on holiday, was behind the Illinois ‘Russian hacker’ scare.

This was compounded by news of the hack at the Texas water plant, where on 20 November a hacker named “prof” gained access to the plant’s systemsusing a three-character default password on an internet-accessed SCADA made by Siemens of Germany. “No damage was done to any machinery; I don’t really like mindless vandalism. It’s stupid and silly. On the other hand, so is connecting your SCADA machinery to the internet,” he wrote on the Pastebin website.

One of PRECYSE’s main approaches to securing systems will be “whitelisting”, a way of ensuring only authorised users obtain access. This is the opposite of the approach used by antivirus software. “Instead of hunting for malicious code, as in an antivirus blacklist, this only lets the known good guys connect,” says security engineer Sakir Sezer at Queens University Belfast in the UK. Unusual behaviour – such as attempting to extract the control codes used to drive equipment – would also mean access is blocked. Deep-packet inspection, normally used to spot copyrighted material on the net, could be harnessed to ensure no attack code is injected. ... 956FB8AF91

In the past two years, hackers have in fact successfully penetrated and extorted multiple utility companies that use SCADA systems, says Alan Paller, director of the SANS Institute, an organization that hosts a crisis center for hacked companies. “Hundreds of millions of dollars have been extorted, and possibly more. It’s difficult to know, because they pay to keep it a secret,” Paller says. “This kind of extortion is the biggest untold story of the cybercrime industry.”

Re: We NSA Now

PostPosted: Tue Feb 12, 2013 10:48 am
by elfismiles

'Google for spies' draws ire from rights groups
Date February 11, 2013
Ryan Gallagher

Riot being used to track someone on Google Earth.

A multinational security firm has secretly developed software capable of tracking people's movements and predicting future behaviour by mining data from social networking websites.

A video obtained by the Guardian reveals how an "extreme-scale analytics" system created by Raytheon, the world's fifth largest defence contractor, can gather vast amounts of information about people from websites including Facebook, Twitter and Foursquare.

Raytheon says it has not sold the software - named Riot, or Rapid Information Overlay Technology - to any clients. But the Massachusetts-based company has acknowledged the technology was shared with US government and industry as part of a joint research and development effort, in 2010, to help build a national security system capable of analysing "trillions of entities" from cyberspace.

The Riot search engine home page.

The power of Riot to harness websites for surveillance offers a rare insight into techniques that have attracted interest from intelligence and national security agencies, at the same time prompting civil liberties and online privacy concerns.

Advertisement Using Riot it is possible to gain a picture of a person's life - their friends, the places they visit charted on a map - in little more than a few clicks of a button.

In the video obtained by the Guardian, Raytheon's "principal investigator" Brian Urch explains that photographs which users post on social networks sometimes contain latitude and longitude details - automatically embedded by smartphones within so-called "exif header data". Riot pulls out this information, showing the location at which the pictures were taken. Riot can display online associations and relationships using Twitter and Facebook and sift GPS location information from Foursquare, a mobile phone app used by more than 25 million people to alert friends of their whereabouts. The Foursquare data can be used to display, in graph form, the top 10 places visited and the times at which they visited them.

Riot being used to monitor someone's check-ins at the gym.

Mining from public websites for law enforcement is considered legal in most countries. But, Ginger McCall, a lawyer at the Washington-based Electronic Privacy Information Centre, said the Raytheon technology raised concerns about how user data could be covertly collected without oversight or regulation.

"Users may be posting information that they believe will be viewed only by their friends, but instead, it is being viewed by government officials or pulled in by data collection services like the Riot search."

Raytheon, which made sales worth an estimated $US25b in 2012, did not want its Riot demonstration video to be revealed on the grounds that it says it shows a "proof of concept" product that has not been sold to any clients.

Jared Adams, a spokesman for Raytheon's intelligence and information systems department, said in an email: "Riot is a big data analytics system design we are working on with industry, national labs and commercial partners to help turn massive amounts of data into useable information to help meet our nation's rapidly changing security needs. Its innovative privacy features are the most robust that we're aware of, enabling the sharing and analysis of data without personally identifiable information being disclosed."

In December, Riot was featured in a new patent Raytheon is pursuing for a system to gather data on people from social networks, blogs and other sources to identify whether they might be a security risk.

In April, Riot was scheduled to be showcased at a US government and industry national security conference for secretive, classified innovations, where it was listed under the category "big data - analytics, algorithms".

The Guardian

Read more: ... 2e75y.html

Re: We NSA Now

PostPosted: Tue Mar 19, 2013 10:05 pm
by General Patton
Course archive for this is still up:
Through this course, students will consider cases describing various organizational struggles: school systems and politicians attempting to implement education reforms; government administrators dealing with an international crisis; technology firms trying to create a company ethos that sustains worker commitment; and even two universities trying to gain international standing by performing a merger.

Each case is full of details and complexity. So how do we make sense of organizations and the challenges they face, let alone develop means of managing them in desired directions? While every detail can matter, some matter more than others. This is why we rely on organizational theories -- to focus our attention and draw out relevant features in a sensible way.

Through this course you will come to see that there is nothing more practical than a good theory. Every week, you’ll learn a different organizational theory, and it will become a lens through which you can interpret concrete organizational situations. Armed with a toolset of theories, you will then be able to systematically identify important features of an organization and the events transforming it – and use the theories to predict which actions will best redirect the organization in a desired direction.

Goes over 10 different theories for analyzing organizations with a focus on case studies from non-profits and governments, good as an introductory course that you can jump right in to. It's important to have several different theories to work with and to test them and apply them to different situations to see how it can change your perspective on them.

Re: We NSA Now

PostPosted: Wed Mar 20, 2013 11:43 am
by General Patton
Some of it is as safe as we think it can be, and some of it is not safe at all. The number one rule of “signals intelligence” is to look for plain text, or signaling information—who is talking to whom. For instance, you and I have been emailing, and that information, that metadata, isn’t encrypted, even if the contents of our messages are. This “social graph” information is worth more than the content. So, if you use SSL-encryption to talk to the OWS server for example, great, they don’t know what you’re saying. Maybe. Let’s assume the crypto is perfect. They see that you’re in a discussion on the site, they see that Bob is in a discussion, and they see that Emma is in a discussion. So what happens? They see an archive of the website, maybe they see that there were messages posted, and they see that the timing of the messages correlates to the time you were all browsing there. They don’t need to know to break a crypto to know what was said and who said it.

Social Organization of Conspiracy
We analyze the social organization of three well-known price-fixing conspiracies in the heavy electrical equipment industry. Although aspects of collusion have been studied by industrial organization economists and organizational criminologists, the organization of conspiracies has remained virtually unexplored Using archival data, we reconstruct the actual communication networks involved in conspiracies in switchgear, transformers, and turbines. We find that the structure of illegal networks is driven primarily by the need to maximize concealment, rather than the need to maximize efficiency. However network structure is also contingent on information-processing requirements imposed by product and market characteristics. Our individual-level model predicts verdict (guilt or innocence), sentence, and fine as functions of personal centrality in the illegal network, network structure, management level, and company size.
A client of ours -- a small, not-for-profit, economic justice organization [EJO] -- used social network analysis [SNA] to assist their city attorney in convicting a group of "slumlords" of various housing violations that the real estate investors had been side-stepping for years. The housing violations, in multiple buildings, included:

raw sewage leaks
multiple tenant children with high lead levels
eviction of complaining tenants
utility liens of six figures

The EJO had been working with local tenants in run-down properties and soon started to notice some patterns. The EJO began to collect public data on the properties with the most violations. As the collected data grew in size, the EJO examined various ways they could visualize the data making it clear and understandable to all concerned. They tried various mind-mapping and organization-charting software but to no avail -- the complex ties they were discovering just made the diagrams hopelessly unreadable. They turned to social network analysis [SNA] to make sense of the complex interconnectivity.
Yet, things were not normal. The EJO discovered that some of the LLC owners were married. As the EJO peeled the onion, more family ties were found within, and between, the LLCs.

Figure 3 shows us that these LLCs were not as separate as they first appeared. The dark red links reveal family ties found in public records. The LLCs were not independent business entities. The business transactions were happening within extended families! A conspiracy was coming into focus.

Image ... 12-E39.pdf
In this study, I examine the effects of corruption on patterns of relations in a well-known case of organizational crime. By contrasting corrupt and non-corrupt projects within an organization, I am able to examine how fraudulent endeavors alter the way individuals mobilize to accomplish a goal. This study examines longitudinal data and couples qualitative coding techniques with social network analysis to understand the effects of corruption on social structure.

In contrast to non-corrupt projects, corrupt networks have lower connectivity, fewer reciprocal relations, and communicate less frequently. These different patterns also hold for between- and within-subject studies. For individuals, corrupt communications, as compared to non-corrupt communications, are less frequent, less likely to be reciprocated, and have reduced transitivity, meaning that message recipients are not as likely to share a communication link. This study highlights the role of content in understanding the emergent properties of communication networks.

Between 2000-2011, the U.S. saw an unprecedented rise in organizational crime, particularly in the form of fraudulent accounting practices by Fortune 500 firms

Why was Bear Sterns and AIG bailed out when Lehman Bros was allowed to fail? How do you quantify too big to fail?
From: ... /index.htm ... -door.html
Mary Jo White’s March 12 confirmation hearing before the Senate Banking Committee on her nomination as chairman of the Securities and Exchange Commission. The highlight was White, the former U.S. attorney in the Southern District of New York turned rainmaker partner at the law firm Debevoise & Plimpton LLP, explaining how her years of legal work for the big Wall Street banks won’t influence what enforcement decisions the SEC makes against the very firms that made her -- and her husband, John White, a top partner at Cravath, Swaine & Moore LLP -- very rich. Their disclosure form indicates they are worth more than $16 million.
There is the assumption that it's the political appointees who run things or change things or are the real power players in DC. My experience has always been that the real power in DC is the persistent class of senior bureaucrats just below the political level. The appointees typically last about 12-to-18 months, getting up to speed for most of that period and--maybe--having some actual impact if they're quite focused in their goals. Otherwise they come and go, leaving nary a trace. They may think they run things and we may hold them ultimately responsible, but the truth is they're more powerless than powerful.

The reality is not the change factor associated with new appointees in an active sense but more in a passive sense: it's not what leadership they bring but what leadership-from-below that they allow.
This is why, when people ask me, "Have you gotten to [some political appointee]?" I tell them that the real goal is to network with the career bureaucrats just below and around that appointee. That's why I love briefing SESers (senior executive service) and top-line general service players in government conferences. Give me a room of 500 of these players, and you've got a serious quorum of influencers.

HQ version: ... rsuade_me/ ... tworks.pdf
In the article “Changing a Culture of Corruption: How Hong Kong's Independent
Commission Against Corruption Succeeded in Furthering a Culture of Lawfulness,” Richard
LaMagna analyzes how Hong Kong was able to change its culture of corruption acceptance.
According to LaMagna, there were four main reasons the Chinese in Hong Kong engaged in
corrupt practices: to get their fair share, to level the playing field, in order to survive, and
because of traditions that viewed certain corrupt practices as acceptable.5 With these reasons in
mind, Hong Kong adopted a three pronged approach to eradicating its corruption problem. The
Independent Commission Against Corruption (ICAC) was responsible for the implementation of
these prongs, which are enforcement, education, and prevention. Since its implementation, “the
ICAC has been successful in changing the culture of corruption.”6 Key to the enforcement aspect
of the ICAC’s plan was removing the responsibility for enforcement (i.e. investigation and
prosecution of suspected corruption) from the police (who were already corrupt), and placing the
responsibility under the purview of the ICAC, which “was empowered to report directly to the
governor…[and] was given broad power of arrest, detention, bail, and search and seizure”.7 Of
course, checks and balances were instituted, with oversight of the ICAC by multiple institutions.
The methods and techniques used to eliminate corruption and to foster a culture of lawfulness are implemented through three types of institutions: civic and school-based education, centers for moral authority, and the media.
Godson argues that the “basic elements of a culture of lawfulness can be built…within one generation.”12 To support this claim, he cites the examples of Hong Kong and Sicily in the 1970’s and 1990’s and also mentions the anti-corruption efforts along the US-Mexico border, Republic of Georgia, and Botswana. Given the examples Godson cites, i.e. countries with a tradition of or precedent for rule of law, the applicability of his approach may be limited in countries that do not have a similar foundation.
LaMagna’s three prongs – enforcement, education, and prevention – along with Godson’s three types of institutions – civic and school based education, centers for moral authority, and the media – all have different leaders at their helm and different methods and pathways for bringing about change most effectively. Using social network analysis, the key players and pathways within the prongs and institutions can be identified, thereby providing negotiators with the decision makers they must influence in order to be successful. Identifying and then surgically targeting these people will likely allow for a more efficient means of combating corruption because negotiators will have to influence fewer people, leaving the indigenous decision makers to deal with the rest of the population with whom they wield more influence.
The seven-element model defines every
type of negotiation as the combination (or absence) of seven different characteristics: interests,
alternatives, options, legitimacy, commitments, communication, and relationships.
18 This framework enables the negotiator with a simple yet robust framework to define and measure success, prepare, consider choices during a negotiation, and review negotiations. The systematic negotiator will recognize that the key to choosing the most effective manner in which to negotiate depends upon the circumstances surrounding the negotiation as well as the desired outcome.
The Three-Layer Model “provides a way for
those working on a partnership interface to understand why behavior that appears
incomprehensible or ill-intentioned might actually be reasonable in an organizational context
different from [one’s] own.”23 This model is a spectrum which illustrates how to understand
behavior from one end to the other; and how to predict behavior from the opposite end, back. At
one end analyzing actions and behaviors is the first step, and analyzing underlying assumptions
and values is the last. From the other end, the order is opposite when attempting to predict
behavior which is slightly more of a preemptive strategy for spotting corrupt practices.24

The study of criminal career paths is necessary to understand the methods of success employed
by high performing criminals. The aim of this paper is to focus this work on the career path of
Jack Herbert who set up and maintained extensive corruption networks between organized crime
groups and police in the Australian state of Queensland. This study builds on Morselli’s work on
the career paths of Sammy Gravano and Howard Marks which demonstrate how understanding
social networks is an essential part of comprehending how organized criminals succeed
. The data
for the current study was taken from the transcripts of the Fitzgerald Commission of Inquiry
which uncovered the extensive and resilient corruption network operated by Herbert. Herbert’s
relationships have been plotted to establish the nature of his operations. The findings indicate that
communication of trust both allows for success and sets the boundaries of a network. Most
importantly, this case study identifies Herbert’s reliance on holding a monopoly as the
cornerstone of his network power and position. This article adds to the literature on criminal
career paths by moving away from a classic organized criminal grouping into the area of police
corruption, and uncovers the distinctive opportunities that this position offers the career criminal

Powerpoint slides and graphs from the Fitzgerald investigation: ... 072219.pdf

Re: We NSA Now

PostPosted: Thu Mar 21, 2013 10:00 am
by elfismiles
Bump and Grind

Re: We NSA Now

PostPosted: Thu Apr 04, 2013 6:53 pm
by General Patton
I've been riding the RP leaderboard as #1 on GlobalCrowd for over 3 months now, so I figured I'd give a status report:

So first let's talk about the red:

Sports, Health, Social:
This is the pop culture category: who is going to win the superbowl, who will win the country music awards, things like that. I didn't do any serious analysis on these types of missions. Not even much work on basic statistics. Interestingly enough my results are close to chance.

Military & Security:
These missions have you predict troop movements, refugee number prediction, invasion timing, causality numbers, prosecution for violations of humans rights issues, that sort of thing.

I relied heavily on 4GW theory for this, along with Analysis of Competing Hypothesis and analysis of history and past trends. Geo-politics has been extremely unstable recently with North Korea, Syria, Libya, the Congo, Sudan, Mali & Iran all seeing enhanced conflict.

A few highlights:
*Always pay attention to what kinds of forces armies are composed of. The ECOWAS forces in Mali for instance is made up almost entirely of Malian troops, so it essentially just the Malian army by a different name. This is important to note when French troops say they are passing command over to ECOWAS, it essentially just means it's going back to the status quo of poorly armed and trained Malian fighters and it's corrupt officer corps.

*There are always more refugees than are officially registered. Being officially registered means you get a shot at aid, the real number of refugees is double or more the registered number. If you want to predict a number over time always look to which countries are already saturated with refugees and if there are neighbors who might open their borders. Keep in mind that refugees likely won't cross conflict areas so that will control which countries they flee too as well. The longer a conflict draws on the less savings/credit refugees will have to draw on so you will see a skyrocketing number if it draws on for several months.

*Pay close attention to tribal/ethnic/religious faultlines, as these are usually how conflicts can be mapped out. Access to OSINT in conflict zones in Africa and the Middle East is extremely sparse, so extrapolating from those faultlines is a must. Investigate people first, then scale out to events, then to ideology.

*Details matter as always. Iran, for instance, has a growing radiomedicine business. That's what the 20% enriched uranium is being used for, they would have to use most of their supply in order to actually build a bomb at current capacity. Supposedly they have also created new types of radiomedicine, though I haven't looked deeply into what they made yet. However that doesn't mean they don't want to build a bomb, remember that Iran's neighbors Israel, Pakistan and India all have their own nuclear weapons. They have also exchanges materials and scientist with North Korea on a limited basis, though sourcing that information is extremely difficult for obvious reasons.

*In many ways operating in an extremely volatile environment with limited access to information makes OSINT nearly impossible.

Science & Technology:
They don't have a lot of these kinds of missions. Progress in science and technology is actually extremely vague and difficult to measure. You may have one GPU that is released and is technically better than the competition, but in reality it's performance is only theoretically higher. Some technology is specialized and there aren't any decent comparisons to make. It's only when there is a wide gap that it becomes obvious which choice was right or wrong.

When I bet on say, SpaceX having a string of successful launches, am I betting on SpaceX's engineers as well as the interpersonal relationships and the structure of the organization of SpaceX more than a given technology. How agile is the organization? How well is their rapid-prototyping structure? What is the incentive system, does it focus on quantity instead of quality?

For example, if I have a staff of coders, do I reward them for the number of bugs that they fix? To what degree do their managers understand whether a bug fix was important or not? Was it just a band-aid fix that can be gamed, or are the managers so inept as to let programmers put bugs into code just so that they can collect more bonuses? Remember that in large structures the appearance of doing work is always more important than doing work.

Business & Economics:

Much of this is a hold over from what I use on Science & Technology and Politics & Policy.

*For predicting something like the FB IPO, you should know that the media loves to tear into a company whenever it does an IPO. The company cannot talk about it's IPO, but media can speculate and lead with headlines in the forms of questions, all business journalism is soft journalism. They cannot possibly sum all of the factors in a meaningful way. Only pay attention to the structure, what message they want to give you, completely ignore the details.

*Look first at the people. Who is the CEO, where was the CEO educated, what kind of social network do they have? Can you listen to some speeches by the CEO and other staff? Look up information on GlassDoor, pay more attention to negative reviews.

*How do they present themselves? Is it a marketing company, a product company, or a service company? Do they make their money off of B2B, consumer or government items?

Groupon and Zynga are marketer companies, built on a fad's that were never entirely productive or useful for the ecosystem. Their products and services require boosting sales, once they've maxed out on adopters and burned enough people the fad will burn out. They boom and bust in about 3 years time. Zynga's innovative strategy was to make shitty games that are copies of other games, then panic when sales decline and try to figure out ways to make more people hooked on them and make removing billing very difficult. Watch out for companies pretending to be useful that only really exist as status symbols.

Apple was a mix of marketer and product company. They had a solid string of early inventions, the Mac iBook was the first laptop with wifi, their computers weren't as customizable but easy to use and made of higher quality materials than Dell computers for example. There were earlier prototypes of iPhone or iPad like devices from other companies, but the screen resolution sucked, the processing speed sucked (2002 processor in a handheld tablet?!), the battery life was horrible, their features were limited or entirely too costly for what little benefit they gave and didn't have reliable wireless. Quality control issues were overcome, lightweight glass invented in 1960 is easy to add once you think of it, and competitor designs were made more aesthetically pleasing, Apple lost it's advantage.

To counter this Apple began courting journalists. Soft bribes are routinely passed around, free swag, paid vacations disguised as conferences, that sort of thing. Building on some truth and a lot of bullshit the Apple image was formed into the psuedo-cult that now exists. Apple also started biting the hand that fed them by putting more and more restrictions on developers, appstore-developer relations can be very fickle.

*Companies that give services to the government exist for reasons of politics as much as anything, find the constituents and the representatives. Ignore market logic and use political models. What does the social network look like? What is the bandwidth of communication and the strength of the connections? Do you have time to create the nodes and edges in Gephi or run them through R? Look at Organizational Analysis by Daniel McFarland on coursera for more.

*As always, use multiple models to judge things. Convergence of multiple models should be watched for.

*What are the demographics of the customers? This is as much an exercise in listening as anything else.

*How mature is the industry, e.g. newspapers versus computer manufacturers?

*Where is the company's products on the Gartner scale?


*Do they rely on early adopters?


*Have they focused on something like optmizing performance at the expense of other areas, leaving a huge gap for a small start-up to exploit? (payment/financial services/law/healthcare start-ups take advantage of this combined with the fact that people see those fields as being too difficult/regulated to create businesses in)

*If you're stuck on something like a stock price, make 3 guesses about where the price could go. Make an upper bound and lower bound for each of those 3 guesses so that you have 9 numbers in total, average those numbers and bet on that range. Start making empirical guesses on the probabilities for events, then investigate when you get them wrong.

*Do the numbers that they release add up?

*How many layers deep and wide did you go into your analysis to make your judgements? Did you connect things from different specialist areas that the majority of people will not connect? Try to make some categories for each set of steps, like fact-checking statistics, running your own statistical analysis, applying models, and see how deep or wide your analysis has gone.

*Look again at the ties between industries and customers, can it be bailed out, e.g. like with banks and the co-risk feedback model? If not, remember that actual events tend to be more extreme and volatile than we can predict.

*Equities markets and start-up's tend to have fat-tails. Very big winners and lots of losers, usually the very big winners don't have a high degree centrality in the beginning. Their ideas may go against the grain or focus on an area of the market that people ignored. As time goes on they gain more connections in the network and their degree increases and they become central.

*Technical Analysis and mathematical models for the market can work solely because people believe it does.

*When building a Bayes model, never assign a probability as zero unless it really is impossible. The variables multiply off of each other and so it will always reduce an event's probability to zero, which has far reaching consequences.

Politics & Policy:

*When debating who will win an election, remember that the media creates candidates for the sole purpose of tearing them down later. See: Herman Cain.

*Look at who is embedded inside of the current bureaucracy. In Egypt many of the same people from the Mubarak regime, e.g. Judges, are still holding their posts, along with people affiliated with the muslim brotherhood. Usually when there is a large change of politics it is only the appearance of such, actual change is typically measured in terms of decades. Same thing with Russia, look at Putin, then go deeper and analyze the bureaucracy that have held power for decades.

*Remember that politicians believe their own bullshit.

*This QZ article has some other useful pieces: ... al-events/

*Learn the subtle differences in how a given government elects or pretends to elect people e.g. the difference between the electoral college in the US versus the coalition system in Israeli politics. No country is just "a democracy".

*Divisions can sprout naturally from people trying to do or be the same thing in a slightly different way. See: Religion, programming languages, ethnicity. Bonus points if the ideology functions as an operating system for the mind, like programming languages and religion.

*Most love is conditional not unconditional so it divides people. Hate unites, anger is the most viral emotion humans have.

*The CIA factbook is always better than wikipedia.

*Watch out for politicians that have something to prove. Maybe 80% of "troop mobilizations" are just scare-mongering shows to distract the citizens of their own countries from internal problems. The other 20% is the gray area of political narcissism, profiteering, stupidity and accidents. We almost blew up the world a few times over computer glitches that showed nuclear missiles incoming, both on the US and Russian side: ... 4bafc0.pdf

*Break regions and demographic groups down. Consider the South Korean and Israeli examples: viewtopic.php?f=33&t=35519&start=15#p491944

*Get on IRC or forums and talk to people from the country in question.


So what stands out is that actual analysis tends to be done very quickly and haphazardly. Models and rigor aren't given much respect in practice. Models are important because it allows you to track how you do things and make improvements in your decision making. In the same way most programmers don't follow documentation procedures or other common sense practices, analysts don't chart out a good analysis of competing hypothesis. Another large problem is giving too much weight to statistics, start-up's pad their numbers or exaggerate their innovations, Spetsnaz training may not be all it's cracked up to be and many financial econometrics exist for the purpose of getting around regulations or managing perceptions by creating obscurity rather than informing an analyst to reality. Many analysts will take up statistics and facts that were conjured from thin air.
At the lowest level, of course, is the raw intelligence report. This report is generally extraordinarily well evaluated and supported. No scholar could really, within the normal limits of national security, ask much more. The source, particularly in CIA-originated reports, is carefully and intelligently described as to his professional knowledge and competence, his outlook, his opportunity to gather the information, and his previous reliability. Not only the date of acquisition of this information but place as well is given. In some reports the rapporteur also provides a field evaluation of the substantive information elicited from the source. The user of this kind of report can easily and effectively apply the canons of evidence in evaluating and testing the information.

But as we move up the ladder of intelligence reports the documentation gets sparser. The NIS (National Intelligence Summary), to use a well-known example, is in effect a scholarly monograph, digesting a great multitude of raw reports. Its total documentation usually consists of a single, very brief paragraph commenting on the general adequacy of the source material. No individual item within the NIS section can be tracked down to a particular source or specific group of sources. As one moves in the NIS from the individual chapter sections to the overall brief, the documentation becomes even more general and less meaningful.

At the more exalted level of the NIE (National Intelligence Estimate), documentation even in the generalized form of comments on sources has usually disappeared altogether. One is forced to rely on the shadings given to “possibly,” “probably,” and “likely” and on other verbal devices for clues as to the quantity and quality of the basic source data. These examples from the NIS and NIE are paralleled in a great many other publications of similar refinement. One may admire the exquisite nuances and marvel at what a burden of knowledge and implicit validation the compressed language of a finished “appreciation” can be forced to carry, but one cannot help being concerned about the conclusions. Upon what foundations do those clever statements rest?

If the final products were at least based upon documented intermediate inputs, the uneasiness might be somewhat less. But in my own experience the “contributions” or inputs, with the exception of certain economic papers, are normally devoid of any specific identification of the kinds and types of reports or other evidence upon which they are based. And in my experience those inputs are often based on other inputs prepared at a lower echelon until at last we reach the analyst with access to the raw data. At the upper level of joint or national discussion and negotiation and compromise, which eventuates in the exquisite nuance, the carefully hedged phrase, or sometimes a dissenting footnote, the remove from the original evidence can be, and often is, considerable.


Re: We NSA Now

PostPosted: Wed Apr 10, 2013 6:31 pm
by General Patton

Psychology of Intelligence Analysis by Richards Heuer ... telNew.pdf

Confronting context effects in intelligence analysis: How can mathematics help? by Keith Devlin ... soning.pdf
& ... tLogic.pdf

A uniform framework for describing and analyzing the modern battlefield ... t_0711.pdf

Also good: What Will Count as Mathematics in 2100: ... n_2100.pdf

Re: We NSA Now

PostPosted: Thu Apr 18, 2013 9:50 pm
by General Patton

Re: We NSA Now

PostPosted: Sun Apr 21, 2013 12:22 am
by General Patton ... nt_mb_1001
Intelligence covers a broad spectrum of disciplines in academia and generally functions to reduce the fog of war within the Department of Defense (DoD). Within the US intelligence community among the 16 primary agencies, intelligence has different scope, different definition.

Thus, when discussing any kind of intelligence it’s important to put it into the context of the agency and the scope. Are you referring to the Central Intelligence Agency (CIA)? If so, are you referring to the Department of Operations (DO)? Or the Department of Intelligence (DI)? At the CIA, for example, the DO is a function of intelligence; however in the DoD J3/G3/S3 or Operations is not a function of intelligence (J2/G2/S2). Operations would continue in the absence of intelligence (perhaps not as effectively) but they would continue.

Most of the time the interface for intelligence with operations in the DoD is the “Two Three” shop. The intelligence operations shop. Hypothetically, this is where Coriolanus meets Jack Murphy. The shop divides further, but that’s a topic for another day.
The Turn

The disciplines of intelligence are human intelligence (HUMINT) or what we classically think of as “spies”. Imagery Intelligence (IMINT) is combined with geographic information systems (GIS). Signals Intelligence (SIGINT), subcomponents of which are communications intelligence (COMINT) and electronic intelligence (ELINT); this is often the most time-sensitive of all the disciplines followed closely by HUMINT.

The most recent entry to our field is a direct result of 9/11, and one I very much appreciate. It is open source intelligence (OSINT). Open source intelligence is basically everything not available using national or military collection efforts. Academia, hard copy archives, media, metadata, data repositories, and blogs are just some examples.

So what brings it all these disciplines together? The fusion analyst.

To put it lightly, a good fusion analyst knows where the bodies are buried. He knows collectors by their collection code and often discusses their findings to gain insight into products. A good analyst knows operators and policy makers by name, meaning he needs to have a good doctrinal understanding of both policy and operations.

Much like one of the two most important lines on an OPORD (Operations Order), you must understand their intent. Despite their nomenclature each of the disciplines produces “data” and that data is a raw “scrape” or “collection effort”. It has not been vetted.

When a fusion analyst puts it together, it becomes “finished intelligence”, the end state of all the individual disciplines. As a fusion analyst I write “evals” on Intelligence Information Reports (IIR) and SIGINT reports, to determine their utility and refine the collections process. The final product is a “fusion” of all inputs into a brief, web page, papers, or network chart or all the above.

In military Intelligence, at the doctrinal joint level (much like Operations) enemy vulnerabilities are broken down using Clausewitz’s “center of gravity” (COG) concepts. This concept was further developed by Dr. Joe Strange, a former professor at the United States Marine Corps (USMC) war college, and last I heard, working at the Counter-IED Operations Integration Center (COIC) and is commonly referred to Systems of Systems Analysis (SoSA).

To understand military intelligence you need to have an intimate understanding of the seven elements of national power: Diplomatic, Information, Military, Economic, Financial, Intelligence, and Law Enforcement. These are all the elements a government can use to exercise its power over both its citizens and other nations.

The doctrinal methodology (as written in Joint Publication 2-0) for looking at DIMEFIL is the Joint Intelligence Processing of the Operational Environment (or JIPOE). This is used at an operational level and referred to as IPOE (or formerly intelligence processing of the battlespace (IPB)). It works hand in hand with the Joint Operational Planning and Execution System (JOPES).

In this analyst’s opinion, to understand your job most effectively you must know how operations work intimately at the tactical, operational, and strategic level. Only then will you be any good to a decision maker. JIPOE categorizes all COG’s into Political, Military, Economic, Social, Infrastructural, and Informational COG’s or PMESII. Notice how these correspond to the elements of national power. This is the relationship between capabilities and vulnerabilities.

What’s important to realize is that COG’s are determined in the “Risks, Assumptions, and Constraints” portion of operational planning. If a decision maker does not have enough information on a node or series of nodes in the COG, he designates this as a Commanders’ Critical Information Requirement (CCIR), and this is then used to build Priority Intelligence Requirements (PIRs). From a bottom up approach: mission, enemy, terrain and weather, troops and support available, time available, civil considerations (METT-TC) becomes IPOE and then this becomes JIPOE.

I am very familiar with tools like Palantir, DCGS-A, M3, etc. and I use all of them. However, that changes from top-down in the military hierarchy. I have all these tools here in CONUS (Continental United States) in a Sensitive Compartmented Information Facility (SCIF).

Once you leave the US and work in TSCIF (T is for tactical) or TOC, you run into bandwidth requirements. Often times at the tactical level I have only had one or two tools available, and then you have to rely on the operator’s and analyst’s best weapon…your brain. You have to create time wheels and CARVER charts from ground up. Not hard, but time-consuming and usually you are running against a clock that counts time in lives.
The Prestige

That’s doctrine. It’s important to understand doctrine… because you need to know it. Then you need to understand how to destroy it, how to make it yours and redevelop it. You need to approach a problem set deemed impossible and reduce it to ruins.

What you just learned was doctrinal intelligence.

My job and what I do currently is to solve “impossible” problems. Problems other analysts have given up on because there is too much data, too little time, or it’s just too hard. My data sets resemble an OLAP (OnLine Analytic Processing) cube.

An OLAP cube is a visual model of three dimensions of data. I am in essence called on to learn a subject as fast as possible, become close to an expert on it, and then analyze and synthesize a solution to a problem.

Bigger problem sets are usually 90 days…smaller ones four days. Recently an epidemiologist we worked with pointed out that what we did in four days would have taken him at least a year, if not two.

I’ve had to become an expert on everything from nuclear and biological weapons to illicit financial networks. What does sharia compliant mean? I can tell you. What does Shor’s algorithm have to do with quantum entanglement? I can tell you.

I use tools (the image to the lower left and lower right are images of visual analytics Starlight creates) that are generally acquired as part of a larger budget (one not available to the public). We use semantic mapping and entity extraction, combined with every piece of data we can get our hands on. Our facilities house entire copies of 10 years of military message traffic. We have complete copies of many SECRET and TOP SECRET intelligence databases joined with open source databases like LiveShips.

We stick it all in a virtual sandboxed environment and see what pops up. What data we can’t get up front we use “cut outs” to acquire. We code API’s and then we take existing programs like Palantir and force them work with others like Starlight.

We use Hadoop as a distributing computing system and we use Field Programmable Gate Arrays (FPGA) to take it outside the wire. We use NoSql to avoid inter-relational databases and just create a cloud. We munge data using Google Refine, and then push it through programs like Future Point’s Starlight. When we are done…we take it to operators. Cyber operators…tactical operators…intelligence officers…and we say “Run with it.”

We’ve helped in personnel recovery (the recovery of Captain Scott Speicher for instance) and discovered efforts by the People’s Republic of China (PRC) to acquire tech via illicit means. This is the new intelligence analyst. This intelligence analyst is a data scientist. It’s an empirical look at big data, sifting through it, and finding those same vulnerabilities as in traditional analysis in the enemy and then taking the fight to them.

The figure on the left are the methodologies I use and the figure on the right are the tools I use.


We try to use as many as possible on any given problem set.

Welcome to spook country. ... tive-unit/
Killing Pablo mentions at one point how when one Presidential administration steps down, that sometimes SOCOM will start freelancing to a certain extent, knowing that it will take the new administration months to play catch up. Killer Elite contains information about how ISA was getting tailed around DC because of Pentagon petty jealousies. As the story goes, one ISA operator cornered his tail in a the bathroom of a fast food joint and put the screws to him until he started to sing like a canary. They had now idea if they were being shadowed by their own people or if it was a terrorist group!

I've shared this site before but I don't recall it being mentioned here, for all things spook related I check out Cryptome.

They have a searchable database and cover every thing from Imint, Sigint, Humint, CIA, OSS, MI5,and KGB, STASI. This site made it's name showing open source Sat images of UK secret military facilities. They have received a lot of pushback from the UK's MoD and even a little from the US DoD. They have some amazing open source data.


Ah, even SOFREP was featured on Cryptome not so long ago. They had our link on their main page. I just read through all those ISA docs. Very interesting to say the least. ... 3/19a.aspx
Machine learning – the ability of computers to understand data, manage results, and infer insights from uncertain information – is the force behind many recent revolutions in computing. Email spam filters, smartphone personal assistants and self-driving vehicles are all based on research advances in machine learning. Unfortunately, even as the demand for these capabilities is accelerating, every new application requires a Herculean effort. Even a team of specially-trained machine learning experts makes only painfully slow progress due to the lack of tools to build these systems.

The Probabilistic Programming for Advanced Machine Learning (PPAML) program was launched to address this challenge. Probabilistic programming is a new programming paradigm for managing uncertain information. By incorporating it into machine learning, PPAML seeks to greatly increase the number of people who can successfully build machine learning applications and make machine learning experts radically more effective. Moreover, the program seeks to create more economical, robust and powerful applications that need less data to produce more accurate results – features inconceivable with today’s technology.

“We want to do for machine learning what the advent of high-level program languages 50 years ago did for the software development community as a whole,” said Kathleen Fisher, DARPA program manager.

Today’s analyst is confined in a T-SCIF (Tactical-Secure Compartmented Information Facility) or SCIF. Information is fed to the analyst over government circuits. The circuits are classified. The analyst will occasionally have an unclassified circuit controlled by the USG.


Use the starfish network. Start with a misattribution system, force analysts to start on the unrestricted internet, and then move up the chain of circuits from SECRET to TOP SECRET. Use sites like, media, social media, open source data repositories. These are your first weapons of choice.

Then, when you run out of valid data…hack. Hack, hack, hack, hack, hax0r, hax0ring. Just do it. Do it now. Back Orifice, E-Bomb, DDoS, bot-nets, malwarez, warez, keygens, brute force, overclock CPU’s and shutdown the fan via killer poke.

If you can’t get it, shut it down.

Use the internet to scrape all your open source data and consider hax0ring as NTM (national technical means). Then when the scrape cycle has completed once, push it up. Push it up the classification ladder and cross-cue the data using the aforementioned tools with national technical means.

Think of the internet as the ocean, then the next level as your first catch, and then the last level the best of the catch. Then push it down. Take that and refine the scrape cycle. Use analytic methodologies.

Consider eliminating the classification system and using obfuscation on the low side. Create 50 different variations of the same message broadcast and then re-broadcast. Tell your receiver the correct variant. 20 ICBMs? Or five?

How can your enemy tell the difference? Which message is “real”? Use the media to hype up different variations of the right message. Yes, I mean actual traffic derived from different sources. Tell the NYT we have 20. Tell the Post we have eight. If you have 52 cards and all of them are aces with each containing a minor change, which ace is the true ace? How long would it take you to tell?

As it stands it may not be any different from the leak-ridden culture of one million cleared personnel you have now. Use the massive information barrage to your advantage and beat your enemy’s OODA loop. Open source warfare is not for the timid, and its encroaching on an entropic system that is archaic. Information always seeks to be free. More rebuttal to come?


Re: We NSA Now

PostPosted: Wed May 08, 2013 8:38 pm
by General Patton

2007 - Untangling the Web - A Guide To Internet Research ... he_Web.pdf

Re: We NSA Now

PostPosted: Tue Apr 22, 2014 2:13 pm
by Wombaticus Rex
Some great Daniel Geer talks/essays:

We Are All Intelligence Officers Now -

On Measurement -

"People in the Loop" - Failsafe or Liability? -

Heartbleed as Metaphor -

PostPosted: Tue Apr 29, 2014 1:04 pm
by Perelandra
^Thank you for those articles. It's too bad the General appears to be on hiatus, because I appreciate his threads and am mining them where I can, although I can't understand much of them.

Re: We NSA Now

PostPosted: Sat Jun 07, 2014 3:21 pm
by cptmarginal
We Are All Intelligence Officers Now -

Wow, I didn't know what Daniel Geer's job specifically was when first reading this. Definitely makes me interpret it a bit differently...

Re: We NSA Now

PostPosted: Mon Feb 09, 2015 8:33 pm
by elfismiles
Norse - IPViking (Live Global CyberAttacks Map)

Published on Sep 8, 2014
Internet Armageddon! The mesmerizing Norse Live Attack Map shows worldwide attacks in real time, as they happen.

DARPA: Nobody's safe on the Internet - CBS News
Video for DARPA Launches Revolutionary, Search Engine That Exposes The Dark Web | VIDEO▶▶ ... -dark-web/
New search engine exposes the "dark web" ... to secure the Internet: Dan Kaufman, a former video ...

Re: We NSA Now

PostPosted: Fri Oct 02, 2015 1:51 am
by General Patton
elfismiles » Mon Feb 09, 2015 7:33 pm wrote:Image
Norse - IPViking (Live Global CyberAttacks Map)

Published on Sep 8, 2014
Internet Armageddon! The mesmerizing Norse Live Attack Map shows worldwide attacks in real time, as they happen.

DARPA: Nobody's safe on the Internet - CBS News
Video for DARPA Launches Revolutionary, Search Engine That Exposes The Dark Web | VIDEO▶▶ ... -dark-web/
New search engine exposes the "dark web" ... to secure the Internet: Dan Kaufman, a former video ...

That map shows honeypots that are being attacked, it's fairly random. Real intrusions by governments tend to use a lot of social engineering up to and including getting spies to turn over data. Which is pitifully easy because the USGOV has practically no counter-intelligence ability left at all. The NATSEC state is falling apart in front of our eyes, all of the wolves are going in for the kill. Hardly a week goes by without a Federal employee learning that China/Russia/Anyone totally owns their identity and work history and can blackmail them at will. God only knows how many refugees/snitches have already gotten killed off by Russia and China. We've lived to see the death of the Republic. Enjoy it.


Wannabe War-On-The-Rocks tryhards are going to tell you that it can be ducktaped together, it's fucked. At best it will smoulder on like a tire fire. I'm not sad to see it go.