I’ve written before about the problems with dialing that happen with our K12 collaborative projects. Here’s one of my really sticky examples from this week.

• School A, one of my schools, can only dial out due to the district firewall.
• Their partner’s school B has an IP plus extension dialing (I think it’s behind a Polycom V2IU). My school couldn’t call it. After some experimentation, I figured out that if our units are registered to our TANDBERG gatekeeper, they can’t call extensions.
• So we tried to unregister, but the unit had an admin password on it that I didn’t have record of. Thankfully the district tech figured that out.
• Sometimes I can put it up on my TANDBERG MPS MCU, but I have it set up with conference 1 as the one conference that people outside can dial into with just the MCU’s IP address. That conference was already in use during the time their program was scheduled. I couldn’t use one of the other conferences to dial out to school B because I can’t call out to an IP plus extension from our bridge unless the bridge unregisters from the gatekeeper & then my other conferences won’t work.
• So I begged School B’s educational service agency to connect with their Polycom MGC. They can connect their school & have my School A’s unit dial in without any gatekeeper problems. So voila, it worked! Whew.

I realize that I could have my MCU set with the Single Dialin Number, but I haven’t had good luck with other schools being able to navigate the menus. So I’m not quite ready to switch to that model.

Here is a second scenario; not quite as complicated.

• My School C is behind a Polycom V2IU and the connection requires an IP plus extension to dial in.
• The partner class, School D, is new to VC. They appear to be set up for dial out only.
• They have a Tandberg 6000 which can’t dial the IP plus extension.
• In this case, my conference 1 is available on the bridge, so that’s how they are connecting. Whew. Another connection solved.

Remember the days of ISDN? Have I forgotten already? or does it seem to you also that dialing was easier then. The teacher/building coordinator in School A mentioned that this is the reason other teachers don’t want to try videoconferencing.

Are you running into the same issues? Do you have any interesting scenarios of trying to make a connection to work?

Recently, I was reading the release notes for the latest version of the TANDBERG MPS MCU software after we upgraded our bridge. I noticed a section about H.239 that indicated that passing the H.239 streams between different types of bridges – the Polycom MGC, Codian, TANDBERG, still doesn’t work consistently. There are known issues.

I’ve written already about other scenarios I’ve seen where H.239 doesn’t work.  The MCUs talking to each other seems to be another area of potential conflict.

So if you have a unit that does H.239, you should always have a back up plan for showing your visuals. I’m thinking specifically of MysteryQuest where we connect a lot of different types of endpoints in the same conference, and even Read Around the Planet.

Note: If you are reading this and it’s a few months or more past the date of posting, please test it out for yourself. I know the vendors are always working on these issues and they may have solved it by the time you read this.

Have you ever found that your videoconference equipment has unexplained packet loss? Do you know how to find out if you are losing packets during a videoconference? Do you know what packets are?

I’m starting to teach my building coordinators how to pay attention to these things. This is the way I explain packet loss to them: Everything on the Internet is broken into little packets to send over the wire. With email, if a packet gets lost along the way, the receiving end asks for it again and eventually you get the whole email. With videoconferencing, if a packet is lost or arrives too late, it’s thrown away because the information is real time. Late information isn’t useful. This is what causes colored blocks on the screen. The information for that block didn’t come through.

Then I show them how to look at the packet loss during a connection. Most of my schools have Polycom VSXs. So I show them how to press the yellow Help button during a call to get the call statistics. (Unless that button has been reprogrammed for another feature. Also can someone reply & share how to do this on other units?)

I tell my building coordinators also that usually they can’t do anything about it and neither can I. However it can give us useful information.

For example, if usually you get 1% of less packet loss and suddenly you’re getting 5-7%, then you should tell your techs. Something might be up with the network. They say that 2% and below is tolerable; but if you get higher than 2% you really start to notice and it can affect the educational quality of the experience. Sometimes the packet loss in a videoconference is an early warning of problems with a router or switch. In this way, VC can be the “canary in the coal mine” for network health.

In two of my buildings this fall, we had some unexplained packet loss. In theory everything else on the network should have meant that they would get a solid 0% packet loss connection. But they were still losing packets to an annoying level.

Come to find out in both situations, the patch cable (also known as the ethernet cable) from the Polycom to the jack in the wall was bad. In one situation I actually saw the cable. It was quite mangled and the cart wheels had rolled over it enough to flatten one section. We swapped out the cable, and voila! 0% packet loss like it should have been.

So, if you have unexplained packet loss, check out your patch cable! Sometimes we forget about the wear and tear on technology in schools. That cable might be due for a change.

I learned a new videoconference troubleshooting term today. “Ear rot” is what you get when you spend hours & hours on the phone with the helpdesk. It’s pretty funny actually, when you’ve spent all day on the phone on a Friday. I’ll refrain from naming the vendor that gave me “ear rot” today. LOL. Problem still isn’t fixed either.

I just keep telling myself, it’s for the kids! When videoconferencing works, it’s such a great experience for the kids!!! Learning experiences are worth the pain.

As I’ve said before, our usage of videoconferencing has grown so much that I am changing policies and procedures so that my districts and buildings can do more videoconferences on their own. This is the only way that we can continue to increase our use of videoconferencing. So there are two new pieces that are almost ready for the new school year.

1. Polycom Import Directory
Yesterday I figured out how to export a directory from my Polycom VSX 7000, edit the CSV file to add many more entries, and upload it to another Polycom VSX 7000. Voila! I’m very pleased with the results. My new directory has all the BCISD numbers they might want to dial; all the numbers for our local buildings; and a set of the most popular content providers we connect to. I may need to make district specific CSVs for a couple of my districts: the two that are behind a V2IU for how they dial each other; and the one that uses internal addresses to dial each other in their district.

2. Scheduling & Troubleshooting Cheatsheets
The other piece is a one page front & back sheet to keep next to the videoconference system or in the cart. It’s pretty specific to our area, so I’m not posting it online. But here are some of the components. The scheduling side has details about our specific scheduling procedures, as well as the unit’s IP address & distance learning coordinator’s name, email & phone. I’m hoping to start having my buildings do their own scheduling for things that are free and that they are connecting on their own.

The title on the back side is: HELP!! Videoconferencing Troubleshooting. It has my phone number and my test unit & office IP addresses. The rest I’ll copy & paste here in case it is useful for you. Many of my buildings don’t have someone who can stay in the room with the teacher while the connection is happening. In some of them the teacher is doing their own dialing as well. So I need ways to provide support quickly and increase the troubleshooting knowledge in the building. You can see that in the handout I’ve made the problem title big & bold so they can easily jump to the section that is causing a problem.

————————————————————————————————————-

TV is Black / I Can’t See

• Check the TV/monitor. Is it on?
• Check the monitor. Is it on the right channel? S-Video for flat screens. Notes: ­­­______________
• If you were able to see the Polycom screen before you connect, then it’s not your TV. A blue or black screen is often a firewall problem on either end of the call. Try our test number (above) to make sure you can connect outside your district on your own. Have the other site try connecting somewhere else too.
• Once in a while the flat screen TVs won’t cooperate. If so, unplug the power, wait, plug it in.

Audio: I Can’t Hear

• Check your TV volume. Check your Polycom volume too. Both should be about in the middle.
• Have the other side check their microphone. If you see a Far End Mute icon, you know their microphone is muted. (This only shows up in point-to-point calls). Ask them to unmute. Tell them you can’t hear. Have them nod their head or wave if they can hear you.

Audio: They Can’t Hear Me

• Check your microphone. Are you muted? Your microphone is muted when the light on it is red. Press the button to unmute.
• Their TV volume might be turned down. You may have to write them notes on a sign to hold up in front of the camera or write on paper under the document camera.

They Can’t See Me

• Have them check their monitor/projector/TV. Is it on & on the right channel?
• It may be a firewall problem on their end. See “I Can’t See” above. Have them try connecting to the BCISD test unit. DON’T give them Janine’s office number. They should see if they get a picture and then call you back again and report. If they can’t see on yours or the ISD’s, they should talk to their tech person.

Call Rejected or Busy Signal or Call Rings & Rings

• If you call and get a “call rejected” error, usually the other person is already in a call.
• If the call rings & rings, usually something in the network between you & the other site is not allowing the call to negotiate. Both sides should try the BCISD test unit. If that works, try the emergency multipoint unit below to make your connection (if you’re in a call right now). If your test call just rings & rings, then schedule with Janine to put the call on the BCISD bridge.

Alerts: What Do They Mean?

• IP Network: If this is down, then you don’t have a live Internet connection. Try another jack.
• Gatekeeper: This is used for multipoint calls and ISDN. If the IP network is up and this is down, don’t worry about it.

When All Else Fails, Reboot or Redial

• If you have a lot of connection problems, sometimes redialing will help clear it up.
• If nothing is working, reboot the camera. Turn the camera off (reach up!!), wait, turn it back on.

Still Can’t Connect? Try the BCISD Emergency Multipoint

• BCISD Emergency Multipoint Polycom: xxx.xxx.xxx.xxx
• This is set up for emergencies only! i.e. you have kids in front of you waiting. First dial in and make sure no one else is already using it. Then give the IP address of the Multipoint Polycom to your partner. Both of you dial in. Have your conference!

————————————————————————————————————-

What kinds of resources do you provide to your buildings to support their use of videoconferencing?

Here are some notes and ramblings from the Internet2 Commons Site Coordinator On-line Workshop today. If you want to attend future sessions, sign up for the email list.

H.239 is NOT there yet. While the tool has potential, I have yet to see it work well in a conference with multiple kinds of units & MCUs. Maybe I’m old fashioned, but give me the old S-Video connection into the codec anytime. It just seems to work consistently across any kind of unit or MCU. To the receiving sites it’s just another camera input.

VC is the canary on your network. It’s always good to have a reminder that while things might work “fine” on a network, videoconferencing may not. VC shows the potential problems and issues on your network.

Get rid of hubs! I know in our elementary schools we still have some hubs in the network path of the videoconference. Something to work on upgrading as time and money allow.

Ethernet duplex mismatch between device and switch. There is a need for collaboration between videoconferencing and networking people to work to solve these types of issues. See this Cisco page and Hostingtech page. It’s best to talk to your networking people and check the duplex settings. If at all possible, stick with auto negotiate.

UDP. The majority of H323 videoconference content is UDP because if packets don’t get there in time they are discarded (vs. TCP where the packet is sent again).

An interesting thought about packet shaping tools. If the system doing the packet shaping is overloaded it can introduce jitter and packet loss because it can’t keep up. So it may not always be a quick fix tool for bandwidth and traffic problems.

Firewalls. An interesting thing about H323 and firewalls is the UDP ports. Usually the devices negotiate with each other which UDP ports to use for video and audio. So think about those calls that start, ring & ring, but never finish negotiating and you know that at least one firewall is involved in the call. The two devices probably can’t negotiate the UDP ports to use. This also explains why if you set your codec to fixed ports for UDP, then sometimes you can’t connect to other sites because their codec might be locked down to different UDP ports.

It’s interesting also that this workshop suggests that using a firewall that “snoops the H323 call set up channels and opens ports for the audio/video as needed.” With my limited experience with firewall traversal systems, I think I agree.

NAT. If you ever see a 10.* or 192.168.* you can know for sure that it’s an internal NAT address that you cannot call from outside the network. If you’re talking to people outside your network, make sure you know how to give them the external address.

Latency (the transmission delay) can be introduced with an overloaded network. There’s the “store-and-forward” delay which is the time it takes the networking devices to receive, process, and resend the packets. It varies based on the network load. Jitter is the variation in latency over time. Someone shared that anything over 50ms becomes a problem.

Packet loss. Numbers to pay attention to. 1% is noticable. 5% is intolerable. How high have you seen? Me – 80%. Pretty bad!!

As participants shared, I thought about all the different network names out there. How many videoconference networks can you name off the top of your head? CANET, MOREnet were two mentioned. Others I can think of are TETN, JANET, K12HSN, to name a few. Leave a comment if you know where those are! Pop quiz!

H323 is “network sensitive.” Ever heard that phrase? It’s an apt description of what we deal with on a regular basis.

Chuckles. They suggested that you shouldn’t make changes to the network without letting everyone know. Yes, wouldn’t it be nice if in little K12 schools we could know when someone changes something on the network!! Usually the techs are so busy that they forget videoconferencing until a problem comes up.

GDS has redundant gatekeepers with geographically dispersed gatekeepers to take over in case of a disaster. So in theory if GDS doesn’t work, it’s probably at your endpoint.

They say that if you want to be a citizen of the world you should join the GDS dialing plan. I tend to agree with this but haven’t been able to implement it successfully yet. But I really want to so that my schools can call JANET schools without going through an MCU. Did you figure out where JANET is yet?

The whole gatekeeper section of this workshop was very helpful. While we’re not really into gatekeepers so much in K12, I do think it’s one potential solution to many of the dialing problems we have.

The workshop also covered MCUs, gateways, etiquette and Internet2 Commons procedures. If you look at the procedures implemented in the Internet2 Commons, they are a great model for any K12 educational service agency.

This was a great workshop and one I recommend for anyone getting started with supporting videoconferences.

Recently someone asked me for suggestions on how to set up videoconferencing “so that we do not tie up our school’s T1 line.” This is certainly a challenge and we all wish for more bandwidth. But when you don’t have it, what can you do?

I have some smaller districts (1500 students or less) using 1 T1 for 2-3 buildings and here are some of the strategies we use:

1. Make sure unitedstreaming cannot be used live. Set it so that teachers have to download ahead of time (i.e. at night).
2. Set QoS (Quality of Service) on the T1 for one videoconference – 384K – to ensure enough bandwidth for the connection. Or lock down the endpoint and QoS to just 128K.
3. Use whatever means necessary/available to make sure only one videoconference is scheduled at a time.
4. Sometimes extreme measures are used as well. One of my buildings makes an announcement over the PA that everyone should get off their computers because a videoconference is happening. In another building, the teachers receive an email ahead of time asking them to stay offline during the videoconference.

In addition, the resources on the Technical page on TWICE are very helpful. What suggestions do you have?

Did you know you can open a Polycom Trouble Ticket online? You can also login and see the status of your tickets as well. Pretty cool. I know many of you have Polycom systems and maybe you know this already. I just learned it today and thought you’d want to know too!