An opaque landscape

I’ve written before about the maddeningly opaque landscape of psychoanalytic training institutes in New York. Even those of us who are psychoanalysts, who’ve navigated that opaque landscape, who’ve selected an institute, trained, and become a member of an analytic institute or society tend not to know much beyond our own institutes and the two or three or five institutes most similar. I trained at CFS, the Contemporary Freudian Society. CFS is a relatively small institute, but it’s one of New York’s oldest, and it’s a “component society” of the International Psychoanalytical Association (the IPA), the international group of institutes founded by Sigmund Freud. The IPA has five or six component societies in New York, and most of my CFS colleagues are familiar with our peers within the IPA. But there are, at last count, somewhere between thirty and forty institutes providing psychoanalytic training in New York. Most of us couldn’t name more than five or six.

For a few years in the mid-2010s, Columbia University (which just today disgracefully bent its knee to the authoritarian, increasingly fascist, administration) provided a tremendous service for those seeking analytic training. It hosted an annual fair for psychoanalytic institutes – a forum at which substantially all (thirty-three) of the psychoanalytic institutes in the New York area (including New Jersey and Long Island) were represented. Each staffed a table, and Columbia put together a catalog listing key facts about all of the institutes.

This was the catalog they distributed at the 2014 fair:

Incredibly useful, it was [imagine the voice of Yoda].

Rendering the opaque transparent

In the years since, I’ve fantasized repeatedly about building an opinionated guide to institutes in New York, about rendering that opaque landscape somewhat more transparent, more legible, for interested prospective candidates. I’ve been thwarted over the years by some combination of distractions from the rest of my life and my own technological limitations. At some point, my fantasy transitioned from being that of presenting to the world a complete and thorough such guide or compendium to presenting instead something a bit more gestural, a bit more flexible – something in the form of a wiki. My fantasy was (or is) that by hosting such a wiki, by putting in the work to make such a thing possible, and by seeding it with some basic data, it might grow into something really useful for a broad audience.

The day that I can do that is rapidly approaching, and I thought rather than letting the perfect be the enemy of the good, I would share my progress as I move forward in this project.

AI’s helpful role in compiling a directory

Artificial intelligence has simplified several different aspects of the project. For example:

Research

Anyone nowadays could go to Google Gemini’s or ChatGPT’s “Deep Research” and ask one of them to create a directory. I did that. I went to them, and to Claude, and to DeepSeek, and tasked them with putting together a simple directory of all the psychoanalytic institutes in New York. The landscape is sufficiently opaque that all of them struggled a bit with this task, but with a little iteration and less than an hour of my time (though over the course of several clock hours, because ChatGPT took over an hour simply to process my initial request), I was able to produce the following deeply flawed table:

Now. Please. Don’t rely on this. This is pretty close to raw AI slop. It could guide you on a bit of a Perplexity or Google jag, so you could get the real deal, but you can’t rely on what’s in here. The table’s flawed. Not all the information in it is correct. I know that. It’s incomplete. The Columbia directory listed 33 institutes! I’m simply showing it to you to show you my trajectory.

OCR and processing of data

I fed the 2014 Columbia catalog into ChatGPT and Claude, and asked them to convert them to text. ChatGPT failed miserably. But Claude did pretty great. Until somehow the conversation exceeded its permitted length about a paragraph before the end. No matter what I did, I couldn’t get Claude to complete the entry for the Women’s Therapy Centre Institute (something I hadn’t previously heard of). ChatGPT came to the rescue, though, able to tackle this discrete problem well enough.

I should say: if the stakes were higher, I would check all this. Did Claude and ChatGPT get all this right? Honestly, I don’t know. I haven’t even read through it once. I don’t care. For my purposes, right now, it’s a good start. You can see what they came up with here:

Research (2)

Somewhere between when I started writing this post and when I finished it, I got off the wait list for Manus – a new AI tool that’s gotten a lot of press lately. (As I understand it, basically, Manus is Claude 3.7 Sonnet plus a host of add-ons that augment its functionality, to impressive effect.)

My first instruction to Manus was similar to the one I’d given ChatGPT and Gemini, previously: “Create a directory of psychoanalytic training institutes in New York City. Include basic contact info as well as info about their training programs.” Manus took about half an hour, and at least appeared to be doing some impressive research. At the end of it, though, what it came up with wasn’t all that much better than the original ChatGPT Deep Research product. You can see it here:

I mean – it was prettier, it was more comprehensive in the data it gathered on each individual institute, but… eight?!? It left off some of the most well known (NYU Postdoc, NIP, ICP, the William Alanson White Institute), not to mention, mine!

My next instruction to Manus, though, was: “That’s a good start. But. I want to give you something that might be helpful. You only listed eight institutes. In 2014, I went to a conference that featured 33 institutes! Please review the catalog from that conference, and update your directory to include all of those.”

Manus good-naturedly replied, “Thank you for providing the Columbia Psychoanalytic Fair program! This is exactly what I needed to create a comprehensive directory. I’ll now:
Extract all the institutes from the PDF
Research their contact information and training program details
Update our directory to include all institutes
Continue developing the website with this expanded data
Deploy the website permanently [I unintentionally asked Manus to make its directory into a website – not yet my intention]
I’ll get started right away and keep you updated on my progress.”

[Manus access is limited. On the free plan – I’ll upgrade soon, I’m reasonably certain – you only get a few “tasks,” and I’m about to run out. But I have the sense that Manus might well be able to take me a good portion of the way to completion of this project….]

At this point, I was feeling pretty hopeful….

And damn – it kinda came through. I say “kinda,” because, well, a lot of the information is wrong. So again – don’t rely on this for anything other than Googling fodder. But still – just having a list of all these institutes in one place is, in and of itself, kind of an incredibly useful thing, no???

Next steps

So my next steps are, a) begin to fact-check the data in this document, initially using AI, and ultimately, using humans, and then b) to try to convert it into some sort of dynamic wiki form! I’m excited. Are you?

You can see where I am at the moment by clicking here. I’d love feedback!