Self organizing map videos: Please watch the videos that the members of the SOMs group have prepared to better explain SOMs and how to build them. Three of these can be found in the Asgn_19_SOMs folder on Box, and the fourth can be viewed here on YouTube https://www.youtube.com/watch?v=zcRCGTIQ03U&feature=youtu.be
Post a PCRC, addressing ALL of the videos. Members of the SOM group, please respond to blog posts.
The Systems Biology of COVID-19 and the SARS-CoV-2 virus. The class will build a foundation that includes the emergence of complexity, simple biological subsystems, their reductionist and equivalent toy and organ-chip models, and the measurements required to specify model architecture and parameters. Applications to biology, physiology, medicine, chemical and biological defense, pharmacology, drug discovery, and toxicology. UGrad: PHYS 240 01 and BME 290B; Grad: PHYS 326 and BME 395C.
0. Knew:
ReplyDeleteI had some understanding about SOMs from previous class discussions. I knew that the goal is to organize the data through some type of similarity criterion.
1. Learned:
Kuniko’s Video: The main point of this video was that SOMs can use an algorithm that produces a mapping that takes a continuous high dimensional space onto a discrete low dimensional space. This mapping is produced through the algorithm which moves data points around based on how close input points are to training points (Maybe?). I am still unsure of the finer details of this process.
Zach’s Video:
I gained more of an understanding of how vectors are changed to produce an SOM and what freedoms we have in how much we change these vectors. I also saw how properties of data vectors at given times as well as different times of acquisition can come into play in creating SOMs. Also, thanks for the shout out.
Amy’s Video:
Great visualization of creating SOMs. I gained more of an understanding of the iterative process of creating a SOM.
Ben’s Video:
I learned about the importance of reducing dimensionality and how it relates to choosing a type of mapping. I also gained more of an understanding of the history/importance of SOMs. I liked seeing these ideas presented in more than just a biological context.
2. Pressing Questions:
Kuniko’s Video: What is the point of the lines connecting the circles in the example of the self-organization process? Also, the example labels O’s as both being inputs and outputs at various points in the example. I am confused as to what the X’s and O’s actually are/where they “live”.
Zach’s Video:
I think I am almost to a good understanding at how SOMs are made. I saw how the colors in the SOM were the same colors as the 5 Kohonen vectors on the board. I also saw how after choosing a vector from the data the yellow vector is moved “closer” to the data vector. I also understand that this is repeated for all of the data (or until convergence?). I am lost as to how this produces the final SOM. I feel like I am almost there, I just need that final jump in understanding. Can you help me out here?
Amy’s Video:
Just like on Zach’s video, I feel like I am almost to the point of having a satisfying amount of understanding of how SOMs are created. In all of the animations in this video, I saw how the data became more organized, but how does this relate to the color organization we see in other SOMs? Also, how does this process change if we are looking at data that changes over time (like gene expression levels)?
Ben’s Video:
Can you choose maps where nodes can have different numbers of neighbors (besides just 4 and 6 for squares and hexagons)? What is the difference between training data and your actual data?
3. Presentation Topic:
One final, careful tutorial through creating an SOM. Something that puts it all together.
4. Thoughts:
Great videos! I am getting closer to understanding these SOMs.
Hi Aaron! In the animations, you'll see the sample input vectors shown as a whole bunch of dots distributed relatively uniformly in the input space. This is the 'shape' of the sample space, so to speak, and its natural clustering state. The neuron weights are either 2D (x, y) or 3D (x, y, z) depending on the example.
DeleteIn the color examples, the neuron weights are 3D (R, B, G). The (R, G, B) weights wiggle across iterations during training just like how (x, y, z) weights wiggle in the animations. It's simply a different way of showing the wiggle process.
Perhaps if you think of it like a heatmap it may make more sense. Here's a link:
https://www.componentone.com/newimages/Products/ScreenShots/StudioSilverlight/C1Chart3D/Chart3D_FloorCeil.png
Actually - try this link:
Deletehttp://lea.verou.me/wp-content/uploads/2014/04/cube-screenshot.png
The training data is the same as your actual data, the training part just means its the data that the map is attempting to model. The final map is the "actual data."
Delete0. KNEW:
ReplyDeleteI only knew about SOMs from the discussions we've had in class. I know that they are used to create low-dimension visual representations of higher dimension data. I knew that during the process of creating an SOM, as one unit is changed, another must be changed in the opposite way.
1. LEARNED:
Ben: I better understood how SOMs perform a mapping of a high-dimensional input into a low-dimensional representation. The example from the paper by Koua and Kraak was a helpful example of an SOM because how strongly countries are related is a straightforward concept. This example did a good job showing how there are different variable associated with each block on an SOM.
Kuniko: I learned that neural networks are organized in layers that each contain a specific "activation function." I learned about unsupervised learning and topology-preserving maps, as well as the 4 stages of an SOM algorithm.
Zach: I learned about classifier maps and how vectors come into play when examining SOMs. He gave a wonderful explanation that gave a different perspective from the other videos.
Amy: I really enjoyed the videos that took you through the entire process of creating an SOM. The visualizations helped me comprehend the process rather than just showing an input and an output.
PRESSING QUESTIONS:
Ben: What is the benefit to using a hexagonal map that has 6 neighbors compared to a square map that only has 4? Or does it depend on the context of the SOM that you are trying to create?
Kuniko: Do all unsupervised learning algorithms create topology-preserving maps?
Zach: It seems like there are a lot of variables that can be changed while building an SOM, but if the complex input data is difficult to understand how can you manipulate these variables to get an output that is expected?
Amy : The final animation with the blue dot and the red blob was extremely helpful, but I am still confused about how the radius of the neighborhood neurons decreased over time other than the program becoming more accurate, or maybe that's supposed to be the takeaway?
PRESENTATION:
A final visualization how these methods and algorithms relate to the colored pixel SOM that was shown during the first two project updates.
THOUGHTS:
Thank you all very much for making these videos. I feel comfortable with the goals of SOMs and I have a much better understanding of how they work.
Hi Jeffrey!
DeleteThe radius of the neighborhood neurons decrease over time because the training process can simply described to have a broad cluster-defining phase, followed by a fine-tuning phase.
The broad cluster-defining phase requires large neighborhoods to be changed after each iteration. That way you have a well defined region that a specific input vector would fall into every time it is called for training.
The fine-tuning phase requires smaller neighborhoods, such that the centroids of each cluster are made to resemble points from the input space as closely as possible. You want to make sure that an input vector consistently falls at one specific location, and that can only be done if you update ever decreasing neighborhoods until you eventually update only one specific neuron.
Hi Jeff,
DeleteI would say no; looking into various unsupervised learning algorithms–Flat Clustering, Hierarchical Clustering, Adaptive Resonance Theory, Competitive Learning–these aren't said to be topology-preserving. Unsupervised learning simply means the algorithm draws inferences from data sets that are unlabeled.
Hey Jeff, good question on squares vs. hexagons. I don't have a solid answer, but I can speculate. Since hexagons have 6 neighbors (vs. 4) and the goal of the SOM is to map the relationship between areas in multidimensional space, a hexagon based map should result in a smoother map since you can show more relationships. Would be worthwhile to see if the MEDI developers have thoughts on squares vs. hexagons.
DeleteColbie Chinowsky
ReplyDeleteReading Assignment 19-SOMs
0. Knew
From our numerous discussions in class, I understood that SOMs allow for visualization of high dimensional data in fewer dimensions.
1. Learned
Kuniko: SOMs are artificial neural networks. Training must be known with SOMs, and neighborhood relationships are preserved, so the nodes that are similar to each other stay close together.
Zach (I admittedly had a hard time hearing the audio in your video, so I may have misheard some things) : Each iteration of the SOM algorithm “self-corrects” to bring the Kohonen vector closer to the input vector, as well as those nearby. One way to make sure that the Kohonen vectors are not unfairly weighted is through vector quantization, which attempts to create a probability distribution. How you spoke assigning colors to vectors was very helpful.
Amy: The “permanment location but variable weight” description of individual neurons was very helpful. The visualization was spot on as well.
Ben: Each node has a set of attributes, including position and weight. Nodes acquire points from the training data.
2. Pressing
Kuniko: Why are output nodes initially given a random weight? What needs to be taken into consideration when choosing initial parameters?
Zach: You talked briefly about using two different SOMs to deal with distinct parts of a dataset. I understand why you would want to you this, but how would one initial the self-organizing maps to assure they would do this?
Amy: How do the maps “know” when to stop? Is it just that the selection of an input doesn’t result in the movement of any neurons?
Ben: What is defined as “training data”?
3. Presentation
Would anyone be up for an “elevator speech” type presentation on SOMs?
4. Thoughts
My understanding of SOMs and how they work was cleared up a lot. While I think ya’ll did a really good job of covering the algorithm itself, and the nitty gritty details, I still find that the “simple” nature of SOMs escapes me; in other words, how would you explain an SOM to a five year old? I do believe that this is possible!
Hi Colbie!
DeleteYou define beforehand the number of iterations you want to conduct during training.
The update magnitude decreases exponentially over time, so eventually you'd be needlessly iterating without making significant changes/wiggles. It's pretty nice to avoid such a scenario, so generally a couple thousand iterations are a good ballpark number to start with.
This comment has been removed by the author.
DeleteRegarding #4:
DeleteSay you want to sculpt a bust, for example of Dr. Wikswo. You begin with a untouched block of clay.
First, you slice off huge hunks of clay with a big wire to define broad regions of the head, neck, and shoulders.
Second, you take a smaller wire and slice off smaller chunks to define the eyes, ears, nose, mouth, etc.
Third, you use a sculpting tool to refine the facial features, making sure that the individual hair strands are shaped, the eyebrows have the proper shape, and the nostrils are nicely circular.
Then you're done! It's not perfect because you're not Michelangelo, but it's as close as you can get.
Hi Colbie,
DeleteYou use random weights to initialize the inputs. Kohonen used random weights, though new methods have emerged since his seminal paper on SOMs (1990). Now PCA is often used to map these weight from the first principal components of the data. This initialization method, however, isn't optimal for all datasets, as that depends on the geometry of the specific data at hand.
Parameters that are important to consider in initialization include σ(0): initial neighborhood size and η(0): initial learning rate.
Hey Colbie,
DeleteThe training data is the same as your actual data, the training part just means its the data that the map is attempting to model. The final map is the "actual data."
As for an elevator pitch, SOM's allow you to easily see the relationships between points in high dimensional space in just two dimensions.
0. Knew:
ReplyDeleteFrom class, I've learned the purpose of SOMs and basic understanding on how they are created. Topological structures to describe SOMs.
1. Learned:
Kuniko: Kohonen networks, "competitive learning", neighborhoods in topological maps, 4 stages in setting up a SOM was made very clear
Zach: Good to see the SOMs formation as vectors rather than arbitrary dots. Learned what can be changed in the formation of an SOM. Using Kernel function to form SOM. Changing initialization step to construct SOM.
Amy: Liked seeing comparison to biological neuronal networks. Helped to understand how this was technique was designed. Demonstration videos were really helpful to visualize what's going on.
Ben: Comparisons/examples were really useful to help understand SOMs.
2. Pressing ?:
Purpose of higher dimensional map? Would this ever be more useful than a 1 or 2 dimensional?
3. Presentation:
Different networks that can be constructed using SOMs to have a real data point/value for each variable (node, neuron,etc). This would help to bring together these videos that were mostly technical with the utility. (Ben's reference was helpful for this).
4. Thoughts: Great job guys. It was nice to hear your different perspectives to round out my understanding of SOMs.
The purpose of SOMs is to acquire a more intuitive representation of your super-high-dimension dataset, and that would typically be in 1D or 2D. I don't know of any cases in which higher dimensional maps were used, but it's possible.
DeleteNot a math answer but more of a data visualization answer; in general 3d plots are to be avoided because people generally view things in 2D (paper or computer screens) and we're not very good at interpreting projections of multidimensional data. Whenever possible stick to 1 or 2 dimensions.
DeleteNick Strayer in BioStats does a great talk on this and other data vis dos and don'ts.
http://nickstrayer.me/visualization_in_r/
Ben Terrones
ReplyDeleteSysBio16_Asgn_19_10_11_2016
0. KNEW:
I knew what we had discussed in previously classes about what SOMs are, why they are used, and a little about how they are created.
1. LEARNED:
In Ben’s video I learned more about why we use SOMs and reduce dimensionality. I also gained a better understanding of distance specifically in the example of dividing up the world. I also have a better understanding of the process of actually creating the map. In Kuniko’s video I learned about what makes SOMs self-organizing. I also learned that SOMs are a type of artificial neural network and what neural networks and topology preserving maps are. In Amy’s video I learned the two ways artificial neural networks can form. I also learned about the four components of self-organization. The animations Amy had were very helpful. In Zach’s video I learned about the independent variables in SOMs that can be manipulated for specific purposes. I also learned about some practical problems with implementing SOMs.
2. PRESSING ?:
For Ben: What would be the reasons for using hexagonal rather than square? Can shapes other than these two be used?
For Kuniko: Do the two important features of topology preserving maps apply to SOMs as a whole?
For Amy: When is supervised learning used to construct artificial neural networks?
For Zach: Since this was the last video I watched, most of my questions had been answered by this point, so I will ask if since making this video you have found an instance where using maps of maps would be helpful?
3. PRESENTATION:
A presentation on some of what Zach hinted at in his video about using SOMs of SOMs, perhaps an example of this or of using multiple SOMs for different parts of a neural network.
4. THOUGHTS:
These were great videos that were very helpful and much better than reading papers.
Yes! I was describing SOMs in the context in which they were initially proposed (Kohonen networks), but all SOMs are topology preserving map; so they i) preserve the relative distant between points and ii) close relationships (i.e. points that are near each other in the input spacE) are preserved in the SOM, such that it can be used as a cluster analysing tool for high-dimensional data.
DeleteAsgn 19, Watch four SOMs video
ReplyDelete0. KNEW
From the previous papers and presentations, I knew that SOMs were a way to reduce the dimensionality of a high-order data sets and also to arrange the data points based on the similarity of their vectors. I remembered that we can use PCA to initially plot the data points on the SOM. I knew that somehow "random" points on the graph were chosen and the distance to this point from the points of the actual data set were compared.
1. LEARNED
Ben: This video helped me understand the need for reducing dimensionality. I really enjoyed and understood the subway system example! The figure of the different health/demographic survey data points helped me to understand how SOMs can be sorted based on many different parameters/variables. I was impressed to see that all of the variables displayed has some sort of correlation/gradient in the SOM.
Kuniko: This presentation finally allowed me to understand what is meant by "neural networks" in data processing, how the algorithms are modeled after the neural connections in cerebral cortex. Also, I understand how each of the components (initialization, competition, cooperation, and adaptation) work in regards to the visual changes of the SOM and the math behind the changes.
Zach: I learned how the math works for moving the coordinates of a SOM vector based on the closest experimental data point. It was interesting to see how Kernel functions can be applied to this topic.
Amy: This presentation helped me understand how the many iterations of the SOM procedure leads to the mapping of the untrained vectors to the data set vectors. I thought it was interesting how fewer points on the map are moved as the map iterates since more and more points are accurately matched to the sample points. The videos in the presentation also helped me understand why the SOM has more points than the sample set.
2. MOST PRESSING QUESTIONS
Ben: Is there an equation based on the number of points in the training data and length of the training data vectors that will tell us how many iterations of the SOM algorithm are needed to resolve the SOM?
Kuniko: What sort of equations are used to complete the "convergence" step in finishing a SOM? How does this differ from the ordering phase?
Zach: Are the "codebook" vectors in you presentation the same as the initial random set of data points put on the blank SOM before they are compared to the data set? Does the light purple vector you drew represent a vector in the actual, experimental data set?
Amy: In the first video of the SOM iterations, why does the map end up only occupying one plane? It appears that the map unfolds to only show changes in the z and y direction, but none in the x direction.
Overall: How do we decide how many nodes we need on the map compared to the number of samples in our experimental data set?
3. PRESENTATION TOPICS
Pros and cons of Kohonen SOMs versus other styles of SOMs (including supervised versus unsupervised maps).
Can we use sample data in relation to time to not only find SOMs with nodes having similar time scales but also to determine causation between nodes based on the time delay between increases/decreases in node values over time?
4. THOUGHTS
I found it interesting how each presenter chose to explain SOMs in different ways and focused on different aspects of the process. This really helped to elucidate the different between the set of nodes on the SOMs versus the set of nodes from the experimental data. I'm still curious as to how the two axes of the SOM develop as the model iterates.
Convergence may be viewed as a measure of how well the model represents the underlying input data; therefore convergence criterion may be viewed as a model selection criterion that can distinguish good models from poor models. There are a number of criterion that can be used to analyze SOM convergence; I'll cover a few general methods rather than specific examples to give you an idea of how this is done. These (unexhaustively) include quantization error (for a given training observation, it is the smallest distance between that training observation and any neuron in the SOM); you may also look at neighborhood stabilities to look at whether relationships that exist in the input data are preserved in the SOM; there are also population-based approaches where the neurons of the map represent one population and the training data represents another population; here you may say the SOM has converged if the neuron and the training data populations appear to be drawn from the same probability distribution.
DeleteGreat question Natalie,
DeleteThere's a few answers, but from what I found in Kohenen's book (Pg 19 of pdf below) you want to aim for ~50 points per node, and then about 500x the number of nodes in the map. For a 25x25 map, thats about ~310,000 itterations. Perhaps that's lowered by initializing with PCA?
http://docs.unigrafia.fi/publications/kohonen_teuvo/MATLAB_implementations_and_applications_of_the_self_organizing_map.pdf
http://www.cs.bham.ac.uk/~jxb/NN/l16.pdf
Stephen Lee
ReplyDeleteClass 23, SOM Group Videos
(0) Knew:
Generally confused about SOMs (likely what precipitated this assignment for all of us). My knowledge doesn’t go much further beyond what we’ve covered in class.
(1) Learned:
Amy- I’m intrigued by the idea of modeling unsupervised networks using physiological topologies (artificial neural networks). Kuniko explained this as well but I watched your video first! The juxtaposition between supervised and unsupervised learning was very helpful.
Ben- I love how your video broke everything down, starting from the simplest quality of reducing dimensionality, and so on. Your explanation of the algorithm and the training process at the end was also useful.
Kuniko- Your visualization of the self-organization process was very helpful.
Zach- Your illustrations helped me understand classifier maps a great deal.
(2) Pressing Questions:
Amy- Could you explain a bit further the comparison you make between PCA and self-assembling networks? How are nodes initially laid out? Does their initial layout affect how the SOM is assembled?
Ben- Does randomizing node distances prior to executing the self-assembly algorithm affect the training of the network? Will the final product look different?
Kuniko- I don’t really have any questions. Your video was very technical and very thorough.
Zach- Maps of maps?
(3) Presentation Topic:
On general principle I have no desire to work beyond what was discussed in these videos until we are all sufficiently competent. Maybe in the future we could explore constructing and modeling an SOM using relevant data?
(4) Thoughts:
I appreciate you all taking the time to flesh some of this stuff out. Using lucid/meaningful examples was very helpful; the scientific community could learn so much from all of us.
PCRC
ReplyDeleteSylvia Morrow
Asgn19_SOMs videos
0. KNEW: Basic ideas of SOMs as presented in class (ie. they are related to PCR, they are a way of organizing and reducing multidimensional data)
1. LEARNED:
--In Ben's presentation I found the slide at 14:20 most helpful. It would be nice to talk about this example in terms of the training algorithm, because I'm still not grasping how that works.
--Amy's presentation was the closest I got to understanding the training algorithm, but was a little too abstract for me to understand how exactly this works in real data.
--Zach's presentation was helpful for understanding what these color regions mean ie. they are kind of like a base set of vectors, but as part of the organization process that base is being modified?
--Kuniko's presentation was super helpful in giving a step-by-step, detailed explanation of what is moving and why.
2. PRESSING ?:
--What was the equation on the board in Zach's presentation?
--I'm starting to lose track of what we're actually learning from SOMs. Some of Ben's examples almost made it seem like we were trying to put so much information together that we were losing useful interpretation?
--Are SOMs basically just a set of weighted averages?
3. PRESENTATION:
--Things that we have learned from using SOMs.
4. THOUGHTS: I think this was a really good way to have the information presented.
0.Knew: I knew that SOMs provided a way to represent multidimensional data in spaces of lower dimension. I knew that they are meant to be artificial neuronal networks. I had an idea that they are supposed to form a topographical map of whatever input pattern of data.
ReplyDelete1.Leared:
Kuniko: with the non-Kohonen networks, I learned more about patterns being presented to the network by input layers. I learned more about further processing taking place in the input layers. I learned about the principle of topological map formation. I learned more about the fourth stage of setting up a SOM, in which the weight equation is applied and the weights of the neurons are updated.
Zach: I learned that you can choose the number of dimensions of the classifier map such that you get the most information from it, though really high dimensions are unlikely. I learned about how the proportion variable can be adjusted depending on how much distance we want to correspond to the change in the variables. I learned that more than one SOM can be used to elucidate different regions of the same network.
Amy: I learned more about what adaptive learning is. I got a better idea through the graphics provided about how neuron training takes place.
Ben: I learned about the Brodmann brain map and its relevance to SOMs, and I got further understanding that each node can acquire several points from the training data.
2.Pressing Questions:
Kuniko: How many common features must the inputs have in order to be grouped together? What does this depend on?
Why are SOMs based on the mammalian cerebral cortex specifically?
How many hidden layers is the supervised learning SOM likely to have?
Zach: How does one determine how many dimensions should be chosen for the classifier map? Can I assume that there is always a direct relationship between the quantity of data and the # of dimensions (increase in the former equals an increase in the latter)?
Amy: I think you mentioned that the neuron locations remain constant while the updating was taking place, but the nodes in the colored graphic seemed to be doing a lot of moving. Can you explain what I am missing?
3.Presentation topic: Like Aaron, I think I would benefit a lot from a brief, comprehensive overview of how to put a SOM together.
4.Thoughts: I really liked Ben’s analogy with the transit system map. I have a much better understanding of how SOMs are created; I understand more of the questions that answer “how,” whereas before, I mainly understood the questioned that answered “what.” Each of the group members making a video was a good idea.
Dionne! The neurons are arranged on a map, where each neuron has a specific (i, j) location on the map. This is always constant.
ReplyDeleteWhat changes is the weight or 'value' of each neuron. In most of the examples in the video, you'll see the neurons wiggling because their weights (x, y, z coordinates) are changing with each update. The map will distort because of the training process, but grid lines help to show that the neurons always maintain their (i, j) position on the map.
0. KNEW
ReplyDeleteI knew that SOM’s were a mathematical tool used to map high-dimensional data (as in gene-expression networks) onto a comprehendible lower-dimensional map (as in a two dimensional heat map).
1. LEARNED
I learned a great deal about the creation of SOM’s…depicting hierarchical clustering as a form of SOM’s helped to contextualize them for me (I feel we hear a lot of “nodes” and “neural networks” and “data points” being thrown around vaguely, so the specificity of this was nice).
-Ben’s presentation provided a solid breakdown of the mathematical algorithm for me.
-Amy’s presentation clearly showed visuals of the graphical process of training and the unraveling of the higher dimensionality of the original data set into a cleaner SOM.
-Kuniko clearly defined what is meant by neural networks in this form of data processing.
-Zach gave a helpful visual of a practical algorithmic approach that a software takes in creating a SOM. I am now clear on the independent variables associated with neuron competition and initializing the neural network, or codebook vectors of n-dimensions in this case.
2. PRESSING ?
I’m still shaky on the mapping process. I understand the concept of hierarchical clustering, but I am missing the bridge between that training of the initialized neural network and the final low-dimensional map.
3. PRESENTATIONS
Practical applications of SOM’s outside gene network visualization and understanding
4. THOUGHTS
I really enjoyed this form of presenting. It was manageable and clear, and I know we can assess any questions in person.
Kelly McGee
ReplyDeleteSOMs
0 (Knew): I had an idea of the method by which SOM's are created, and that the "color" or assigned value of each data point was changed, but I was not sure how it was done.
1 (Learned): I learned, from Kuniko and Amy, that the input map is adjusted in a stepwise process, taking each actual data point at a time and adjusting the input map accordingly, until the input map matches the data. Not only that, but in doing so, this creates a map that, in 1 or 2 dimensions, represents higher-dimensional data.
2 (Questions): In Zach's presentation, he mentions that if a data point is exactly between two input field vectors, the algorithm will try to move both vectors closer, and this will create messiness so a "window" is needed to ensure this doesn't happen. How does this window work?
3 (Presentation): See #2.
4 (Comments): Very useful videos that really increased my understanding of SOMs.